[cf-dev] Incubation Proposal: CredHub (credential manager)

classic Classic list List threaded Threaded
16 messages Options
Reply | Threaded
Open this post in threaded view
|

[cf-dev] Incubation Proposal: CredHub (credential manager)

Dan Jahner
Hello Everyone, 

Pivotal would like to propose to the Extensions PMC a new incubation project focusing on credential management in Cloud Foundry. This product may be used in a Cloud Foundry environment to centralize and secure credential generation, storage, lifecycle management and access. 

Project name: CredHub
Proposed Project Lead: Dan Jahner (Pivotal)
Proposed Scope: See “Proposed Scope” in the proposal 
Development Operating Model: Pairing Model 
Technical Approach: See “Basic Architecture” and “BOSH Manifest Implementation” in the proposal 
Initial team committed: 6 engineers from Pivotal 


Please let me know if you have any questions. 

Thanks, 
Dan Jahner 
Reply | Threaded
Open this post in threaded view
|

[cf-dev] Re: Incubation Proposal: CredHub (credential manager)

John Feminella-2
Questions:
  • To what extent are CredHub's use cases and architecture covered (or not) by a combination of something like Hashicorp's Vault and integration effort? (I'm not singling out Vault specifically here, just using it as a representative member of a class.)
  • People who use CF want the properties outlined by CredHub, but CredHub doesn't exist yet. What do those people do instead? To what extent are their current solutions sufficient or deficient?
best,
~ jf
--
John Feminella
Advisory Platform Architect
✉ · [hidden email]
t · @jxxf



On Fri, Dec 16, 2016 6:15 PM, Dan Jahner [hidden email] wrote:
Hello Everyone, 

Pivotal would like to propose to the Extensions PMC a new incubation project focusing on credential management in Cloud Foundry. This product may be used in a Cloud Foundry environment to centralize and secure credential generation, storage, lifecycle management and access. 

Project name: CredHub
Proposed Project Lead: Dan Jahner (Pivotal)
Proposed Scope: See “Proposed Scope” in the proposal 
Development Operating Model: Pairing Model 
Technical Approach: See “Basic Architecture” and “BOSH Manifest Implementation” in the proposal 
Initial team committed: 6 engineers from Pivotal 


Please let me know if you have any questions. 

Thanks, 
Dan Jahner 
Reply | Threaded
Open this post in threaded view
|

[cf-dev] Re: Incubation Proposal: CredHub (credential manager)

David Illsley
In reply to this post by Dan Jahner
It seems like the initial focus is solving the operator problems (via BOSH) rather than app developer problems. Do you have any sketches of how it might be used for application secrets (broker and user provided)?

On Fri, Dec 16, 2016 at 11:15 PM, Dan Jahner <[hidden email]> wrote:
Hello Everyone, 

Pivotal would like to propose to the Extensions PMC a new incubation project focusing on credential management in Cloud Foundry. This product may be used in a Cloud Foundry environment to centralize and secure credential generation, storage, lifecycle management and access. 

Project name: CredHub
Proposed Project Lead: Dan Jahner (Pivotal)
Proposed Scope: See “Proposed Scope” in the proposal 
Development Operating Model: Pairing Model 
Technical Approach: See “Basic Architecture” and “BOSH Manifest Implementation” in the proposal 
Initial team committed: 6 engineers from Pivotal 


Please let me know if you have any questions. 

Thanks, 
Dan Jahner 

Reply | Threaded
Open this post in threaded view
|

[cf-dev] Re: Re: Incubation Proposal: CredHub (credential manager)

Allan Beck
In reply to this post by John Feminella-2
I am supportive of the proposal. We have our own credentials store implementation to encrypt the service credentials passed to applications (so devops staff can’t see credentials to production services). CredHub will be a useful addition to CF and will allow us to use this community component rather than our bespoke implementation.

Regards,
Allan.


On 19 Dec 2016, at 06:42, John Feminella <[hidden email]> wrote:

Questions:
  • To what extent are CredHub's use cases and architecture covered (or not) by a combination of something like Hashicorp's Vault and integration effort? (I'm not singling out Vault specifically here, just using it as a representative member of a class.)
  • People who use CF want the properties outlined by CredHub, but CredHub doesn't exist yet. What do those people do instead? To what extent are their current solutions sufficient or deficient?
best,
~ jf
--
John Feminella
Advisory Platform Architect
✉ · [hidden email]
t · @jxxf



On Fri, Dec 16, 2016 6:15 PM, Dan Jahner [hidden email] wrote:
Hello Everyone, 

Pivotal would like to propose to the Extensions PMC a new incubation project focusing on credential management in Cloud Foundry. This product may be used in a Cloud Foundry environment to centralize and secure credential generation, storage, lifecycle management and access. 

Project name: CredHub
Proposed Project Lead: Dan Jahner (Pivotal)
Proposed Scope: See “Proposed Scope” in the proposal 
Development Operating Model: Pairing Model 
Technical Approach: See “Basic Architecture” and “BOSH Manifest Implementation” in the proposal 
Initial team committed: 6 engineers from Pivotal 


Please let me know if you have any questions. 

Thanks, 
Dan Jahner 

Reply | Threaded
Open this post in threaded view
|

[cf-dev] Re: Re: Incubation Proposal: CredHub (credential manager)

Justin Smith
In reply to this post by David Illsley
It makes sense to build CredHub for many reasons. A few that come to mind quickly are below.

1) The service must start and restart without human intervention. This immediately means the key encrypting key resides in a Hardware Security Module (HSM) of some kind. We think this is a first-class feature and should be part of CredHub open source.
2) CloudFoundry runs in production in some of the most restricted environments in the world. These environments tend towards favoring crypto stacks they have seen and reviewed before. The Java ecosystem contains several. The Go ecosystem will eventually get there, but it isn't right now. We're fully aware there are tons of great Go apps (including large chunks of CloudFoundry) that perform crypto today. We received credible feedback that showed a Java preference for this type of application.
3) Authorization is an important part of credential management in CF. Authorization can be an incredibly difficult problem, and an authorization implementation can be very hard to change after it is built. We didn't see anything available that had the right feature set. We could have bolted something in front of another product, but we decided it made more sense to build something new.

MVP certainly requires hitting a threshold of capabilities, but it also requires leaving runway for what's ahead.

App secrets are a mainline scenario. Let's take it as a given that apps must have access to plaintext secrets. From there, the discussion centers on two questions: 1) what's the chain of custody of the secret and what has access to it, and 2) how is the secret made available to the app. (1) is all about CredHub and (2) is mostly about Diego. We'll engage in a discussion on these in the weeks ahead.
Reply | Threaded
Open this post in threaded view
|

[cf-dev] Re: Re: Re: Incubation Proposal: CredHub (credential manager)

Wayne E. Seguin-2
Most of our customers are using Vault already for secrets management and would prefer to keep doing so as their final "at-rest" store.

For our customers it would therefore work best if CredHub can use Vault.

With this idea, can we design CredHub using interchangeable backends (Perhaps a plugin API for backends) so that our customers can continue to use Vault as the final "at-rest" backend key/credentials store?

On Mon, Dec 19, 2016 at 8:45 PM, Justin Smith <[hidden email]> wrote:
It makes sense to build CredHub for many reasons. A few that come to mind quickly are below.

1) The service must start and restart without human intervention. This immediately means the key encrypting key resides in a Hardware Security Module (HSM) of some kind. We think this is a first-class feature and should be part of CredHub open source.
2) CloudFoundry runs in production in some of the most restricted environments in the world. These environments tend towards favoring crypto stacks they have seen and reviewed before. The Java ecosystem contains several. The Go ecosystem will eventually get there, but it isn't right now. We're fully aware there are tons of great Go apps (including large chunks of CloudFoundry) that perform crypto today. We received credible feedback that showed a Java preference for this type of application.
3) Authorization is an important part of credential management in CF. Authorization can be an incredibly difficult problem, and an authorization implementation can be very hard to change after it is built. We didn't see anything available that had the right feature set. We could have bolted something in front of another product, but we decided it made more sense to build something new.

MVP certainly requires hitting a threshold of capabilities, but it also requires leaving runway for what's ahead.

App secrets are a mainline scenario. Let's take it as a given that apps must have access to plaintext secrets. From there, the discussion centers on two questions: 1) what's the chain of custody of the secret and what has access to it, and 2) how is the secret made available to the app. (1) is all about CredHub and (2) is mostly about Diego. We'll engage in a discussion on these in the weeks ahead.



--
Reply | Threaded
Open this post in threaded view
|

[cf-dev] Re: Re: Re: Re: Incubation Proposal: CredHub (credential manager)

Travis McPeak
I see how getting UAA to interface correctly to somethung like Vault may be difficult.  Aside from that and the language stack, what are primary diffetences with other mature key management systems?

Crypto (and surrounding implementations) are notoriously difficult to get right.  Are we planning to have any kind of third party audit?


On Tue, Dec 20, 2016, 6:05 AM Wayne E. Seguin <[hidden email]> wrote:
Most of our customers are using Vault already for secrets management and would prefer to keep doing so as their final "at-rest" store.

For our customers it would therefore work best if CredHub can use Vault.

With this idea, can we design CredHub using interchangeable backends (Perhaps a plugin API for backends) so that our customers can continue to use Vault as the final "at-rest" backend key/credentials store?

On Mon, Dec 19, 2016 at 8:45 PM, Justin Smith <[hidden email]> wrote:
It makes sense to build CredHub for many reasons. A few that come to mind quickly are below.

1) The service must start and restart without human intervention. This immediately means the key encrypting key resides in a Hardware Security Module (HSM) of some kind. We think this is a first-class feature and should be part of CredHub open source.
2) CloudFoundry runs in production in some of the most restricted environments in the world. These environments tend towards favoring crypto stacks they have seen and reviewed before. The Java ecosystem contains several. The Go ecosystem will eventually get there, but it isn't right now. We're fully aware there are tons of great Go apps (including large chunks of CloudFoundry) that perform crypto today. We received credible feedback that showed a Java preference for this type of application.
3) Authorization is an important part of credential management in CF. Authorization can be an incredibly difficult problem, and an authorization implementation can be very hard to change after it is built. We didn't see anything available that had the right feature set. We could have bolted something in front of another product, but we decided it made more sense to build something new.

MVP certainly requires hitting a threshold of capabilities, but it also requires leaving runway for what's ahead.

App secrets are a mainline scenario. Let's take it as a given that apps must have access to plaintext secrets. From there, the discussion centers on two questions: 1) what's the chain of custody of the secret and what has access to it, and 2) how is the secret made available to the app. (1) is all about CredHub and (2) is mostly about Diego. We'll engage in a discussion on these in the weeks ahead.



Reply | Threaded
Open this post in threaded view
|

[cf-dev] Re: Re: Incubation Proposal: CredHub (credential manager)

Dan Jahner
In reply to this post by David Illsley
Hi David -

You are correct - our initial use case if focused on operator concerns. An additional proposal that focuses on application secrets is forthcoming and will involve the larger group of teams needed to make that successful.
Reply | Threaded
Open this post in threaded view
|

[cf-dev] Re: Re: Re: Re: Incubation Proposal: CredHub (credential manager)

Dan Jahner
In reply to this post by Wayne E. Seguin-2
Hi Wayne -

When you say that your customers would prefer to use Vault as their "at-rest" store, do you know the motivation driving that preference? I assume the concern is primarily focused on the encryption, not the data storage itself (which Vault delegates to standard data stores, e.g. Consul, MySQL, etc.).

CredHub is being developed to provide a pluggable encryption provider interface, which allows users to select the appropriate provider based on their needs. By example, most customers in high security environments would select a hardware security module to perform these cryptography operations.

An integration with Vault doesn’t make sense in this context, because interfacing with the entire Vault codebase to leverage only its encryption features would create an inferior experience to implementing these algorithms natively with well-respected Java libraries. If your customers’ concern is specifically the algorithm, you’ll be happy to know that both the internal software and HSM client providers in CredHub will support AES256-GCM, which is the same algorithm used by Vault.

Please let me know if you have any additional thoughts or concerns.
Reply | Threaded
Open this post in threaded view
|

[cf-dev] Re: Re: Re: Incubation Proposal: CredHub (credential manager)

Mathias Essenpreis-2
In reply to this post by Dan Jahner
Hi Dan,
thumps up for the proposal.

Thinking about application secrets. You state to also cover "Conceal service credentials from the Cloud Controller". Assuming you mean to encrypt e.g. passwords that are stored in the env of an application, right? So I assume this will help applications already to securely store their secrets.

Is my assumption right?
Reply | Threaded
Open this post in threaded view
|

[cf-dev] Re: Re: Re: Re: Incubation Proposal: CredHub (credential manager)

Dan Jahner
Hi Mathias - 

(Copying my reply in the doc for visibility) Correct, our initial plan is for CredHub to store the service credentials. This feature work is a separate initiative and a proposal will be sent out to the community for comment before it is implemented. 

We will not focus on applications directly leveraging CredHub beyond service credentials for this work, but it will move us closer to enabling that use case.

On Wed, Jan 4, 2017 at 12:38 AM Mathias Essenpreis <[hidden email]> wrote:
Hi Dan,
thumps up for the proposal.

Thinking about application secrets. You state to also cover "Conceal service credentials from the Cloud Controller". Assuming you mean to encrypt e.g. passwords that are stored in the env of an application, right? So I assume this will help applications already to securely store their secrets.

Is my assumption right?
Reply | Threaded
Open this post in threaded view
|

[cf-dev] Re: Re: Re: Re: Re: Incubation Proposal: CredHub (credential manager)

Daniel Jones
Hi all,

CredHub sounds awesome.


Is CredHub going to be a proprietary value-add of the PCF distribution? Or is it going to be OS for all to use, and owned by the Foundation?

Regards,
Daniel Jones - CTO
+44 (0)79 8000 9153
EngineerBetter Ltd - UK Cloud Foundry Specialists

On 4 January 2017 at 19:52, Dan Jahner <[hidden email]> wrote:
Hi Mathias - 

(Copying my reply in the doc for visibility) Correct, our initial plan is for CredHub to store the service credentials. This feature work is a separate initiative and a proposal will be sent out to the community for comment before it is implemented. 

We will not focus on applications directly leveraging CredHub beyond service credentials for this work, but it will move us closer to enabling that use case.

On Wed, Jan 4, 2017 at 12:38 AM Mathias Essenpreis <[hidden email]> wrote:
Hi Dan,
thumps up for the proposal.

Thinking about application secrets. You state to also cover "Conceal service credentials from the Cloud Controller". Assuming you mean to encrypt e.g. passwords that are stored in the env of an application, right? So I assume this will help applications already to securely store their secrets.

Is my assumption right?

Reply | Threaded
Open this post in threaded view
|

[cf-dev] Re: Re: Re: Re: Re: Re: Incubation Proposal: CredHub (credential manager)

Dan Jahner
Hi Daniel, 

I'm glad you are excited for CredHub. 

The repos are currently in pivotal-cf as a first step to open sourcing them and moving them to the foundation. As soon as we get the official approval on the incubation request, they will be moved to the appropriate CFF org.

Thanks, 
Dan

On Fri, Feb 10, 2017 at 3:46 AM Daniel Jones <[hidden email]> wrote:
Hi all,

CredHub sounds awesome.


Is CredHub going to be a proprietary value-add of the PCF distribution? Or is it going to be OS for all to use, and owned by the Foundation?

Regards,
Daniel Jones - CTO
<a href="tel:+44%207980%20009153" value="+447980009153" class="gmail_msg" target="_blank">+44 (0)79 8000 9153
EngineerBetter Ltd - UK Cloud Foundry Specialists

On 4 January 2017 at 19:52, Dan Jahner <[hidden email]> wrote:
Hi Mathias - 

(Copying my reply in the doc for visibility) Correct, our initial plan is for CredHub to store the service credentials. This feature work is a separate initiative and a proposal will be sent out to the community for comment before it is implemented. 

We will not focus on applications directly leveraging CredHub beyond service credentials for this work, but it will move us closer to enabling that use case.

On Wed, Jan 4, 2017 at 12:38 AM Mathias Essenpreis <[hidden email]> wrote:
Hi Dan,
thumps up for the proposal.

Thinking about application secrets. You state to also cover "Conceal service credentials from the Cloud Controller". Assuming you mean to encrypt e.g. passwords that are stored in the env of an application, right? So I assume this will help applications already to securely store their secrets.

Is my assumption right?

Reply | Threaded
Open this post in threaded view
|

[cf-dev] Re: Re: Re: Re: Re: Re: Re: Incubation Proposal: CredHub (credential manager)

Daniel Jones
Cool, good stuff.

Regards,
Daniel Jones - CTO
+44 (0)79 8000 9153
EngineerBetter Ltd - UK Cloud Foundry Specialists

On 10 February 2017 at 17:48, Dan Jahner <[hidden email]> wrote:
Hi Daniel, 

I'm glad you are excited for CredHub. 

The repos are currently in pivotal-cf as a first step to open sourcing them and moving them to the foundation. As soon as we get the official approval on the incubation request, they will be moved to the appropriate CFF org.

Thanks, 
Dan

On Fri, Feb 10, 2017 at 3:46 AM Daniel Jones <[hidden email]> wrote:
Hi all,

CredHub sounds awesome.


Is CredHub going to be a proprietary value-add of the PCF distribution? Or is it going to be OS for all to use, and owned by the Foundation?

Regards,
Daniel Jones - CTO
<a href="tel:+44%207980%20009153" value="+447980009153" class="m_-1718056664188137349gmail_msg" target="_blank">+44 (0)79 8000 9153
EngineerBetter Ltd - UK Cloud Foundry Specialists

On 4 January 2017 at 19:52, Dan Jahner <[hidden email]> wrote:
Hi Mathias - 

(Copying my reply in the doc for visibility) Correct, our initial plan is for CredHub to store the service credentials. This feature work is a separate initiative and a proposal will be sent out to the community for comment before it is implemented. 

We will not focus on applications directly leveraging CredHub beyond service credentials for this work, but it will move us closer to enabling that use case.

On Wed, Jan 4, 2017 at 12:38 AM Mathias Essenpreis <[hidden email]> wrote:
Hi Dan,
thumps up for the proposal.

Thinking about application secrets. You state to also cover "Conceal service credentials from the Cloud Controller". Assuming you mean to encrypt e.g. passwords that are stored in the env of an application, right? So I assume this will help applications already to securely store their secrets.

Is my assumption right?


Reply | Threaded
Open this post in threaded view
|

[cf-dev] Re: Re: Re: Re: Re: Re: Re: Re: Incubation Proposal: CredHub (credential manager)

Michael Maximilien-2
Thanks all. We will be moving for a vote on CredHub this week. 

Therefore it's important for all to chime in (if you have additional questions or comments) on this proposal. Silence means all good.

Please use the Google Docs that Dan initially sent. Best,

max

On Tue, Feb 14, 2017 at 8:34 AM, Daniel Jones <[hidden email]> wrote:
Cool, good stuff.

Regards,
Daniel Jones - CTO
<a href="tel:+44%207980%20009153" value="+447980009153" target="_blank">+44 (0)79 8000 9153
EngineerBetter Ltd - UK Cloud Foundry Specialists

On 10 February 2017 at 17:48, Dan Jahner <[hidden email]> wrote:
Hi Daniel, 

I'm glad you are excited for CredHub. 

The repos are currently in pivotal-cf as a first step to open sourcing them and moving them to the foundation. As soon as we get the official approval on the incubation request, they will be moved to the appropriate CFF org.

Thanks, 
Dan

On Fri, Feb 10, 2017 at 3:46 AM Daniel Jones <[hidden email]> wrote:
Hi all,

CredHub sounds awesome.


Is CredHub going to be a proprietary value-add of the PCF distribution? Or is it going to be OS for all to use, and owned by the Foundation?

Regards,
Daniel Jones - CTO
<a href="tel:+44%207980%20009153" value="+447980009153" class="m_7477670582767465307m_-1718056664188137349gmail_msg" target="_blank">+44 (0)79 8000 9153
EngineerBetter Ltd - UK Cloud Foundry Specialists

On 4 January 2017 at 19:52, Dan Jahner <[hidden email]> wrote:
Hi Mathias - 

(Copying my reply in the doc for visibility) Correct, our initial plan is for CredHub to store the service credentials. This feature work is a separate initiative and a proposal will be sent out to the community for comment before it is implemented. 

We will not focus on applications directly leveraging CredHub beyond service credentials for this work, but it will move us closer to enabling that use case.

On Wed, Jan 4, 2017 at 12:38 AM Mathias Essenpreis <[hidden email]> wrote:
Hi Dan,
thumps up for the proposal.

Thinking about application secrets. You state to also cover "Conceal service credentials from the Cloud Controller". Assuming you mean to encrypt e.g. passwords that are stored in the env of an application, right? So I assume this will help applications already to securely store their secrets.

Is my assumption right?





--
Reply | Threaded
Open this post in threaded view
|

[cf-dev] Re: Incubation Proposal: CredHub (credential manager)

Michael Maximilien-2
In reply to this post by Dan Jahner
Hi, all,

As discussed in the CAB call today, I would like to move the CredHub proposal [1] to move to the next steps in order to become a CF incubator.

As per our process, since we are in the Review state, unless there are any pending comments or objections I will move for a vote. Deadline to voice your opinion is tomorrow midnight PDT.

So please, if you have any comment or questions about CredHub please respond here or add to the proposal doc. Dan will respond directly.

If none then the vote should occur soon after and I will report here on result.

Thanks for your time. Best,

max

cf-extensions pmc lead
ibm cloud labs
silicon valley, ca


On Fri, Dec 16, 2016 at 3:15 PM, Dan Jahner <[hidden email]> wrote:
Hello Everyone, 

Pivotal would like to propose to the Extensions PMC a new incubation project focusing on credential management in Cloud Foundry. This product may be used in a Cloud Foundry environment to centralize and secure credential generation, storage, lifecycle management and access. 

Project name: CredHub
Proposed Project Lead: Dan Jahner (Pivotal)
Proposed Scope: See “Proposed Scope” in the proposal 
Development Operating Model: Pairing Model 
Technical Approach: See “Basic Architecture” and “BOSH Manifest Implementation” in the proposal 
Initial team committed: 6 engineers from Pivotal 


Please let me know if you have any questions. 

Thanks, 
Dan Jahner 



--