[deployment] [oslo] [ansible] [tripleo] [kolla] [helm] Configuration management with etcd / confd

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
39 messages Options
12
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[deployment] [oslo] [ansible] [tripleo] [kolla] [helm] Configuration management with etcd / confd

Emilien Macchi-4
Following-up the session that we had in Boston:
https://etherpad.openstack.org/p/BOS-forum-future-of-configuration-management

Here's an update on where we are and what is being done.


== Machine Readable Sample Config

Ben's spec has been merged: https://review.openstack.org/#/c/440835/
And also the code which implements it: https://review.openstack.org/#/c/451081/
He's now working on the documentation on how to use the feature.

$ oslo-config-generator --namespace keystone --formal yaml > keystone.yaml

Here's an example of the output for Keystone config: https://clbin.com/EAfFM
This feature was asked at the PTG, and it's already done!


== Pluggable drivers for oslo.config

Doug's spec has been well written and the feedback from Summit and the
review was taken in account: https://review.openstack.org/#/c/454897/
It's now paused because we think we could use confd (with etcd driver)
to generate configuration files.

Imagine the work done by Ben in Machine Readable Sample Config, that
would allow us to generate Confd templates for all services (Keystone,
Nova, etc) via a tool provided in oslo.config with all the options
available for a namespace.
We could have packaging builds (e.g. RDO distgit) using the tooling
when building packages so we could ship confd templates in addition of
ini configuration files.
When services would start (e.g. in containers), confd would generate
configuration files from the templates that is part of the container,
and read the values from etcd.

The benefit of doing this, is that a very little work is required in
oslo.config to make this happen (only a tool to generate confd
templates). It could be a first iteration.
Another benefit is that confd will generate a configuration file when
the application will start. So if etcd is down *after* the app
startup, it shouldn't break the service restart if we don't ask confd
to re-generate the config. It's good for operators who were concerned
about the fact the infrastructure would rely on etcd. In that case, we
would only need etcd at the initial deployment (and during lifecycle
actions like upgrades, etc).

The downside is that in the case of containers, they would still have
a configuration file within the container, and the whole goal of this
feature was to externalize configuration data and stop having
configuration files.


== What's next

I see 2 short-term actions that we can work on:

1) Decide if whether or not confd solution would be acceptable for a
start. I'm asking Kolla, TripleO, Helm, Ansible projects if they would
be willing to use this feature. I'm also asking operators to give
feedback on it.

2) Investigate how to expose parameters generated by Ben's work on
Machine Readable Sample Config to the users (without having to
manually maintain all options) - I think this has to be solved on the
installers side, but I might be wrong; and also investigate how to
populate parameters data into etcd. This tool could be provided by
oslo.config probably.



Any feedback from folks working on installers or from operators would
be more than welcome!

Thanks,
--
Emilien Macchi

__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: [hidden email]?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [deployment] [oslo] [ansible] [tripleo] [kolla] [helm] Configuration management with etcd / confd

Bogdan Dobrelya-2
On 06.06.2017 18:08, Emilien Macchi wrote:

> Following-up the session that we had in Boston:
> https://etherpad.openstack.org/p/BOS-forum-future-of-configuration-management
>
> Here's an update on where we are and what is being done.
>
>
> == Machine Readable Sample Config
>
> Ben's spec has been merged: https://review.openstack.org/#/c/440835/
> And also the code which implements it: https://review.openstack.org/#/c/451081/
> He's now working on the documentation on how to use the feature.
>
> $ oslo-config-generator --namespace keystone --formal yaml > keystone.yaml
>
> Here's an example of the output for Keystone config: https://clbin.com/EAfFM
> This feature was asked at the PTG, and it's already done!
>

Great progress, well done!

>
> == Pluggable drivers for oslo.config
>
> Doug's spec has been well written and the feedback from Summit and the
> review was taken in account: https://review.openstack.org/#/c/454897/
> It's now paused because we think we could use confd (with etcd driver)
> to generate configuration files.
>
> Imagine the work done by Ben in Machine Readable Sample Config, that
> would allow us to generate Confd templates for all services (Keystone,
> Nova, etc) via a tool provided in oslo.config with all the options
> available for a namespace.
> We could have packaging builds (e.g. RDO distgit) using the tooling
> when building packages so we could ship confd templates in addition of
> ini configuration files.
> When services would start (e.g. in containers), confd would generate
> configuration files from the templates that is part of the container,
> and read the values from etcd.

This sounds like a plan to start following just immediately :-)

>
> The benefit of doing this, is that a very little work is required in
> oslo.config to make this happen (only a tool to generate confd
> templates). It could be a first iteration.

And that's really great as we need no to reimplement confd for oslo.config.

> Another benefit is that confd will generate a configuration file when
> the application will start. So if etcd is down *after* the app
> startup, it shouldn't break the service restart if we don't ask confd
> to re-generate the config. It's good for operators who were concerned
> about the fact the infrastructure would rely on etcd. In that case, we
> would only need etcd at the initial deployment (and during lifecycle
> actions like upgrades, etc).
>
> The downside is that in the case of containers, they would still have
> a configuration file within the container, and the whole goal of this
> feature was to externalize configuration data and stop having
> configuration files.

It doesn't look a strict requirement. Those configs may (and should) be
bind-mounted into containers, as hostpath volumes. Or, am I missing
something what *does* make embedded configs a strict requirement?..

>
>
> == What's next
>
> I see 2 short-term actions that we can work on:
>
> 1) Decide if whether or not confd solution would be acceptable for a
> start. I'm asking Kolla, TripleO, Helm, Ansible projects if they would
> be willing to use this feature. I'm also asking operators to give
> feedback on it.
>
> 2) Investigate how to expose parameters generated by Ben's work on
> Machine Readable Sample Config to the users (without having to
> manually maintain all options) - I think this has to be solved on the
> installers side, but I might be wrong; and also investigate how to
> populate parameters data into etcd. This tool could be provided by
> oslo.config probably.
>
>
>
> Any feedback from folks working on installers or from operators would
> be more than welcome!
>
> Thanks,
>


--
Best regards,
Bogdan Dobrelya,
Irc #bogdando

__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: [hidden email]?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [deployment] [oslo] [ansible] [tripleo] [kolla] [helm] Configuration management with etcd / confd

Emilien Macchi-4
In reply to this post by Emilien Macchi-4
On Tue, Jun 6, 2017 at 6:08 PM, Emilien Macchi <[hidden email]> wrote:

> Following-up the session that we had in Boston:
> https://etherpad.openstack.org/p/BOS-forum-future-of-configuration-management
>
> Here's an update on where we are and what is being done.
>
>
> == Machine Readable Sample Config
>
> Ben's spec has been merged: https://review.openstack.org/#/c/440835/
> And also the code which implements it: https://review.openstack.org/#/c/451081/
> He's now working on the documentation on how to use the feature.
>
> $ oslo-config-generator --namespace keystone --formal yaml > keystone.yaml
>
> Here's an example of the output for Keystone config: https://clbin.com/EAfFM
> This feature was asked at the PTG, and it's already done!
>
>
> == Pluggable drivers for oslo.config
>
> Doug's spec has been well written and the feedback from Summit and the
> review was taken in account: https://review.openstack.org/#/c/454897/
> It's now paused because we think we could use confd (with etcd driver)
> to generate configuration files.
>
> Imagine the work done by Ben in Machine Readable Sample Config, that
> would allow us to generate Confd templates for all services (Keystone,
> Nova, etc) via a tool provided in oslo.config with all the options
> available for a namespace.

I'm also wondering if we could use oslo-config-generate directly to
generate confd templates, with a new format. So we would have ini,
yaml, json and confd.
"confd" format would be useful when building rpms that we ship in containers.
"yaml" format would be useful for installers to expose the options
directly to the User Interface, so we know which params OpenStack
provide and we could re-use the data to push it into etcd.

Would it make sense?

> We could have packaging builds (e.g. RDO distgit) using the tooling
> when building packages so we could ship confd templates in addition of
> ini configuration files.
> When services would start (e.g. in containers), confd would generate
> configuration files from the templates that is part of the container,
> and read the values from etcd.
>
> The benefit of doing this, is that a very little work is required in
> oslo.config to make this happen (only a tool to generate confd
> templates). It could be a first iteration.
> Another benefit is that confd will generate a configuration file when
> the application will start. So if etcd is down *after* the app
> startup, it shouldn't break the service restart if we don't ask confd
> to re-generate the config. It's good for operators who were concerned
> about the fact the infrastructure would rely on etcd. In that case, we
> would only need etcd at the initial deployment (and during lifecycle
> actions like upgrades, etc).
>
> The downside is that in the case of containers, they would still have
> a configuration file within the container, and the whole goal of this
> feature was to externalize configuration data and stop having
> configuration files.
>
>
> == What's next
>
> I see 2 short-term actions that we can work on:
>
> 1) Decide if whether or not confd solution would be acceptable for a
> start. I'm asking Kolla, TripleO, Helm, Ansible projects if they would
> be willing to use this feature. I'm also asking operators to give
> feedback on it.
>
> 2) Investigate how to expose parameters generated by Ben's work on
> Machine Readable Sample Config to the users (without having to
> manually maintain all options) - I think this has to be solved on the
> installers side, but I might be wrong; and also investigate how to
> populate parameters data into etcd. This tool could be provided by
> oslo.config probably.
>
>
>
> Any feedback from folks working on installers or from operators would
> be more than welcome!
>
> Thanks,
> --
> Emilien Macchi



--
Emilien Macchi

__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: [hidden email]?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [deployment] [oslo] [ansible] [tripleo] [kolla] [helm] Configuration management with etcd / confd

Doug Hellmann-2

On Jun 7, 2017, at 7:20 AM, Emilien Macchi <[hidden email]> wrote:

On Tue, Jun 6, 2017 at 6:08 PM, Emilien Macchi <[hidden email]> wrote:
Following-up the session that we had in Boston:
https://etherpad.openstack.org/p/BOS-forum-future-of-configuration-management

Here's an update on where we are and what is being done.


== Machine Readable Sample Config

Ben's spec has been merged: https://review.openstack.org/#/c/440835/
And also the code which implements it: https://review.openstack.org/#/c/451081/
He's now working on the documentation on how to use the feature.

$ oslo-config-generator --namespace keystone --formal yaml > keystone.yaml

Here's an example of the output for Keystone config: https://clbin.com/EAfFM
This feature was asked at the PTG, and it's already done!


== Pluggable drivers for oslo.config

Doug's spec has been well written and the feedback from Summit and the
review was taken in account: https://review.openstack.org/#/c/454897/
It's now paused because we think we could use confd (with etcd driver)
to generate configuration files.

Imagine the work done by Ben in Machine Readable Sample Config, that
would allow us to generate Confd templates for all services (Keystone,
Nova, etc) via a tool provided in oslo.config with all the options
available for a namespace.

I'm also wondering if we could use oslo-config-generate directly to
generate confd templates, with a new format. So we would have ini,
yaml, json and confd.
"confd" format would be useful when building rpms that we ship in containers.
"yaml" format would be useful for installers to expose the options
directly to the User Interface, so we know which params OpenStack
provide and we could re-use the data to push it into etcd.

Would it make sense?

I did think about making oslo-config-generator also take the YAML file as input instead of scanning plugins, and then including all the output formats in the single command. I haven’t looked to see how much extra complexity that would add.


We could have packaging builds (e.g. RDO distgit) using the tooling
when building packages so we could ship confd templates in addition of
ini configuration files.
When services would start (e.g. in containers), confd would generate
configuration files from the templates that is part of the container,
and read the values from etcd.

The benefit of doing this, is that a very little work is required in
oslo.config to make this happen (only a tool to generate confd
templates). It could be a first iteration.
Another benefit is that confd will generate a configuration file when
the application will start. So if etcd is down *after* the app
startup, it shouldn't break the service restart if we don't ask confd
to re-generate the config. It's good for operators who were concerned
about the fact the infrastructure would rely on etcd. In that case, we
would only need etcd at the initial deployment (and during lifecycle
actions like upgrades, etc).

The downside is that in the case of containers, they would still have
a configuration file within the container, and the whole goal of this
feature was to externalize configuration data and stop having
configuration files.


== What's next

I see 2 short-term actions that we can work on:

1) Decide if whether or not confd solution would be acceptable for a
start. I'm asking Kolla, TripleO, Helm, Ansible projects if they would
be willing to use this feature. I'm also asking operators to give
feedback on it.

2) Investigate how to expose parameters generated by Ben's work on
Machine Readable Sample Config to the users (without having to
manually maintain all options) - I think this has to be solved on the
installers side, but I might be wrong; and also investigate how to
populate parameters data into etcd. This tool could be provided by
oslo.config probably.



Any feedback from folks working on installers or from operators would
be more than welcome!

Thanks,
--
Emilien Macchi



-- 
Emilien Macchi

__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: [hidden email]?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: [hidden email]?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [Openstack-operators] [deployment] [oslo] [ansible] [tripleo] [kolla] [helm] Configuration management with etcd / confd

Emilien Macchi-4
On Wed, Jun 7, 2017 at 3:31 PM, Doug Hellmann <[hidden email]> wrote:

>
> On Jun 7, 2017, at 7:20 AM, Emilien Macchi <[hidden email]> wrote:
>
> On Tue, Jun 6, 2017 at 6:08 PM, Emilien Macchi <[hidden email]> wrote:
>
> Following-up the session that we had in Boston:
> https://etherpad.openstack.org/p/BOS-forum-future-of-configuration-management
>
> Here's an update on where we are and what is being done.
>
>
> == Machine Readable Sample Config
>
> Ben's spec has been merged: https://review.openstack.org/#/c/440835/
> And also the code which implements it:
> https://review.openstack.org/#/c/451081/
> He's now working on the documentation on how to use the feature.
>
> $ oslo-config-generator --namespace keystone --formal yaml > keystone.yaml
>
> Here's an example of the output for Keystone config: https://clbin.com/EAfFM
> This feature was asked at the PTG, and it's already done!
>
>
> == Pluggable drivers for oslo.config
>
> Doug's spec has been well written and the feedback from Summit and the
> review was taken in account: https://review.openstack.org/#/c/454897/
> It's now paused because we think we could use confd (with etcd driver)
> to generate configuration files.
>
> Imagine the work done by Ben in Machine Readable Sample Config, that
> would allow us to generate Confd templates for all services (Keystone,
> Nova, etc) via a tool provided in oslo.config with all the options
> available for a namespace.
>
>
> I'm also wondering if we could use oslo-config-generate directly to
> generate confd templates, with a new format. So we would have ini,
> yaml, json and confd.
> "confd" format would be useful when building rpms that we ship in
> containers.
> "yaml" format would be useful for installers to expose the options
> directly to the User Interface, so we know which params OpenStack
> provide and we could re-use the data to push it into etcd.
>
> Would it make sense?
>
>
> I did think about making oslo-config-generator also take the YAML file as
> input instead of scanning plugins, and then including all the output formats
> in the single command. I haven’t looked to see how much extra complexity
> that would add.

Do you mean taking the YAML file that we generate with Ben's work
(which would include the parameters values, added by some other
tooling maybe)?

I see 2 options at least:

* Let installers to feed etcd with the parameters by using this etcd
namespace $CUSTOM_PREFIX + /project/section/parameter (example
/node1/keystone/DEFAULT/debug).
  And patch oslo.config to be able to generate confd templates with
all the options (and ship the template in the package)
  I like this option because it provides a way for operators to learn
about all possible options in the configuration, with documentation
and default values.

* Also let installers to feed etcd but use a standard template like
you showed me last week (credits to you for the code):
http://paste.openstack.org/show/2KZUQsWYpgrcG2K8TDcE/
   I like this option because nothing has to be done in oslo.config,
since we use a standard template for all OpenStack configs (see the
paste ^)

Thoughts?

>
> We could have packaging builds (e.g. RDO distgit) using the tooling
> when building packages so we could ship confd templates in addition of
> ini configuration files.
> When services would start (e.g. in containers), confd would generate
> configuration files from the templates that is part of the container,
> and read the values from etcd.
>
> The benefit of doing this, is that a very little work is required in
> oslo.config to make this happen (only a tool to generate confd
> templates). It could be a first iteration.
> Another benefit is that confd will generate a configuration file when
> the application will start. So if etcd is down *after* the app
> startup, it shouldn't break the service restart if we don't ask confd
> to re-generate the config. It's good for operators who were concerned
> about the fact the infrastructure would rely on etcd. In that case, we
> would only need etcd at the initial deployment (and during lifecycle
> actions like upgrades, etc).
>
> The downside is that in the case of containers, they would still have
> a configuration file within the container, and the whole goal of this
> feature was to externalize configuration data and stop having
> configuration files.
>
>
> == What's next
>
> I see 2 short-term actions that we can work on:
>
> 1) Decide if whether or not confd solution would be acceptable for a
> start. I'm asking Kolla, TripleO, Helm, Ansible projects if they would
> be willing to use this feature. I'm also asking operators to give
> feedback on it.
>
> 2) Investigate how to expose parameters generated by Ben's work on
> Machine Readable Sample Config to the users (without having to
> manually maintain all options) - I think this has to be solved on the
> installers side, but I might be wrong; and also investigate how to
> populate parameters data into etcd. This tool could be provided by
> oslo.config probably.
>
>
>
> Any feedback from folks working on installers or from operators would
> be more than welcome!
>
> Thanks,
> --
> Emilien Macchi
>
>
>
>
> --
> Emilien Macchi
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: [hidden email]?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
> _______________________________________________
> OpenStack-operators mailing list
> [hidden email]
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>



--
Emilien Macchi

__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: [hidden email]?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [Openstack-operators] [deployment] [oslo] [ansible] [tripleo] [kolla] [helm] Configuration management with etcd / confd

Doug Hellmann-2
Excerpts from Emilien Macchi's message of 2017-06-07 16:42:13 +0200:

> On Wed, Jun 7, 2017 at 3:31 PM, Doug Hellmann <[hidden email]> wrote:
>>
>> On Jun 7, 2017, at 7:20 AM, Emilien Macchi <[hidden email]> wrote:
>>
>> On Tue, Jun 6, 2017 at 6:08 PM, Emilien Macchi <[hidden email]> wrote:
>>
>> Following-up the session that we had in Boston:
>> https://etherpad.openstack.org/p/BOS-forum-future-of-configuration-management
>>
>> Here's an update on where we are and what is being done.
>>
>>
>> == Machine Readable Sample Config
>>
>> Ben's spec has been merged: https://review.openstack.org/#/c/440835/
>> And also the code which implements it:
>> https://review.openstack.org/#/c/451081/
>> He's now working on the documentation on how to use the feature.
>>
>> $ oslo-config-generator --namespace keystone --formal yaml > keystone.yaml
>>
>> Here's an example of the output for Keystone config: https://clbin.com/EAfFM
>> This feature was asked at the PTG, and it's already done!
>>
>>
>> == Pluggable drivers for oslo.config
>>
>> Doug's spec has been well written and the feedback from Summit and the
>> review was taken in account: https://review.openstack.org/#/c/454897/
>> It's now paused because we think we could use confd (with etcd driver)
>> to generate configuration files.
>>
>> Imagine the work done by Ben in Machine Readable Sample Config, that
>> would allow us to generate Confd templates for all services (Keystone,
>> Nova, etc) via a tool provided in oslo.config with all the options
>> available for a namespace.
>>
>>
>> I'm also wondering if we could use oslo-config-generate directly to
>> generate confd templates, with a new format. So we would have ini,
>> yaml, json and confd.
>> "confd" format would be useful when building rpms that we ship in
>> containers.
>> "yaml" format would be useful for installers to expose the options
>> directly to the User Interface, so we know which params OpenStack
>> provide and we could re-use the data to push it into etcd.
>>
>> Would it make sense?
>>
>>
>> I did think about making oslo-config-generator also take the YAML file as
>> input instead of scanning plugins, and then including all the output formats
>> in the single command. I haven’t looked to see how much extra complexity
>> that would add.
>
> Do you mean taking the YAML file that we generate with Ben's work
> (which would include the parameters values, added by some other
> tooling maybe)?
>
> I see 2 options at least:
>
> * Let installers to feed etcd with the parameters by using this etcd
> namespace $CUSTOM_PREFIX + /project/section/parameter (example
> /node1/keystone/DEFAULT/debug).
>  And patch oslo.config to be able to generate confd templates with
> all the options (and ship the template in the package)
>  I like this option because it provides a way for operators to learn
> about all possible options in the configuration, with documentation
> and default values.
>
> * Also let installers to feed etcd but use a standard template like
> you showed me last week (credits to you for the code):
> http://paste.openstack.org/show/2KZUQsWYpgrcG2K8TDcE/
>   I like this option because nothing has to be done in oslo.config,
> since we use a standard template for all OpenStack configs (see the
> paste ^)
>
> Thoughts?

There are 2 problems with using the generic template.

1. In order for confd to work, you have to give it a list of all of the
  keys in etcd that it should monitor, and that list is
  application-specific.

2. Not all of our configuration values are simple strings or numbers.
  We have options for managing lists of values, and there is even
  an Opt class for loading a dictionary for some reason. So,
  rendering the value in the template will depend on the type of
  the option.

Given those constraints, it makes sense to generate a custom template
for each set of options. We need to generate the confd file anyway, and
the template can have the correct logic for rendering mult-value
options.

One further problem I don't know how to address yet is the applications
that use dynamic sections in configuration files. I think Cinder
is still the primary example of this, but other apps may use that
ability.  I don't know how to tell confd that it needs to look at
the keys in those groups, since we don't know the names in advance.

Doug

__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: [hidden email]?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [deployment] [oslo] [ansible] [tripleo] [kolla] [helm] Configuration management with etcd / confd

Flavio Percoco-2
In reply to this post by Bogdan Dobrelya-2
On 07/06/17 12:04 +0200, Bogdan Dobrelya wrote:

>On 06.06.2017 18:08, Emilien Macchi wrote:
>> Another benefit is that confd will generate a configuration file when
>> the application will start. So if etcd is down *after* the app
>> startup, it shouldn't break the service restart if we don't ask confd
>> to re-generate the config. It's good for operators who were concerned
>> about the fact the infrastructure would rely on etcd. In that case, we
>> would only need etcd at the initial deployment (and during lifecycle
>> actions like upgrades, etc).
>>
>> The downside is that in the case of containers, they would still have
>> a configuration file within the container, and the whole goal of this
>> feature was to externalize configuration data and stop having
>> configuration files.
>
>It doesn't look a strict requirement. Those configs may (and should) be
>bind-mounted into containers, as hostpath volumes. Or, am I missing
>something what *does* make embedded configs a strict requirement?..
mmh, one thing I liked about this effort was possibility of stop bind-mounting
config files into the containers. I'd rather find a way to not need any
bindmount and have the services get their configs themselves.

Flavio


--
@flaper87
Flavio Percoco

__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: [hidden email]?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

signature.asc (879 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [deployment] [oslo] [ansible] [tripleo] [kolla] [helm] Configuration management with etcd / confd

Flavio Percoco-2
On 08/06/17 18:23 +0200, Flavio Percoco wrote:

>On 07/06/17 12:04 +0200, Bogdan Dobrelya wrote:
>>On 06.06.2017 18:08, Emilien Macchi wrote:
>>>Another benefit is that confd will generate a configuration file when
>>>the application will start. So if etcd is down *after* the app
>>>startup, it shouldn't break the service restart if we don't ask confd
>>>to re-generate the config. It's good for operators who were concerned
>>>about the fact the infrastructure would rely on etcd. In that case, we
>>>would only need etcd at the initial deployment (and during lifecycle
>>>actions like upgrades, etc).
>>>
>>>The downside is that in the case of containers, they would still have
>>>a configuration file within the container, and the whole goal of this
>>>feature was to externalize configuration data and stop having
>>>configuration files.
>>
>>It doesn't look a strict requirement. Those configs may (and should) be
>>bind-mounted into containers, as hostpath volumes. Or, am I missing
>>something what *does* make embedded configs a strict requirement?..
>
>mmh, one thing I liked about this effort was possibility of stop bind-mounting
>config files into the containers. I'd rather find a way to not need any
>bindmount and have the services get their configs themselves.
Probably sent too early!

If we're not talking about OpenStack containers running in a COE, I guess this
is fine. For k8s based deployments, I think I'd prefer having installers
creating configmaps directly and use that. The reason is that depending on files
that are in the host is not ideal for these scenarios. I hate this idea because
it makes deployments inconsistent and I don't want that.

Flavio

--
@flaper87
Flavio Percoco

__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: [hidden email]?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

signature.asc (879 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [deployment] [oslo] [ansible] [tripleo] [kolla] [helm] Configuration management with etcd / confd

Michał Jastrzębski
On 8 June 2017 at 09:27, Flavio Percoco <[hidden email]> wrote:

> On 08/06/17 18:23 +0200, Flavio Percoco wrote:
>>
>> On 07/06/17 12:04 +0200, Bogdan Dobrelya wrote:
>>>
>>> On 06.06.2017 18:08, Emilien Macchi wrote:
>>>>
>>>> Another benefit is that confd will generate a configuration file when
>>>> the application will start. So if etcd is down *after* the app
>>>> startup, it shouldn't break the service restart if we don't ask confd
>>>> to re-generate the config. It's good for operators who were concerned
>>>> about the fact the infrastructure would rely on etcd. In that case, we
>>>> would only need etcd at the initial deployment (and during lifecycle
>>>> actions like upgrades, etc).
>>>>
>>>> The downside is that in the case of containers, they would still have
>>>> a configuration file within the container, and the whole goal of this
>>>> feature was to externalize configuration data and stop having
>>>> configuration files.
>>>
>>>
>>> It doesn't look a strict requirement. Those configs may (and should) be
>>> bind-mounted into containers, as hostpath volumes. Or, am I missing
>>> something what *does* make embedded configs a strict requirement?..
>>
>>
>> mmh, one thing I liked about this effort was possibility of stop
>> bind-mounting
>> config files into the containers. I'd rather find a way to not need any
>> bindmount and have the services get their configs themselves.
>
>
> Probably sent too early!
>
> If we're not talking about OpenStack containers running in a COE, I guess
> this
> is fine. For k8s based deployments, I think I'd prefer having installers
> creating configmaps directly and use that. The reason is that depending on
> files
> that are in the host is not ideal for these scenarios. I hate this idea
> because
> it makes deployments inconsistent and I don't want that.

Well, I disagree. If we're doing this we're essentially getting rid of
"files" at all. It might actually be easier to handle from COE than
configmap, as configmap has to be generated and when you get to host
specific things it's quite a pain to handle. I, for one, would happily
use cantral DB for config options if we define schema correctly.

That being said defining schema correctly is quite a challenge. Few
hard cases I see right now can be found in single use case - PCI
Passthrough

1. I have multiple PCI devices in host, I need to specify list of them
2. PCI buses differes host to host, I need to specify groups of hosts
that will share same bus configuration and reflect that in service
config

Maybe we should gather few of hard use cases like that and make sure
we can address them in our config schema?

>
> Flavio
>
> --
> @flaper87
> Flavio Percoco
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: [hidden email]?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>

__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: [hidden email]?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [deployment] [oslo] [ansible] [tripleo] [kolla] [helm] Configuration management with etcd / confd

Doug Hellmann-2
In reply to this post by Flavio Percoco-2
Excerpts from Flavio Percoco's message of 2017-06-08 18:27:51 +0200:

> On 08/06/17 18:23 +0200, Flavio Percoco wrote:
> >On 07/06/17 12:04 +0200, Bogdan Dobrelya wrote:
> >>On 06.06.2017 18:08, Emilien Macchi wrote:
> >>>Another benefit is that confd will generate a configuration file when
> >>>the application will start. So if etcd is down *after* the app
> >>>startup, it shouldn't break the service restart if we don't ask confd
> >>>to re-generate the config. It's good for operators who were concerned
> >>>about the fact the infrastructure would rely on etcd. In that case, we
> >>>would only need etcd at the initial deployment (and during lifecycle
> >>>actions like upgrades, etc).
> >>>
> >>>The downside is that in the case of containers, they would still have
> >>>a configuration file within the container, and the whole goal of this
> >>>feature was to externalize configuration data and stop having
> >>>configuration files.
> >>
> >>It doesn't look a strict requirement. Those configs may (and should) be
> >>bind-mounted into containers, as hostpath volumes. Or, am I missing
> >>something what *does* make embedded configs a strict requirement?..
> >
> >mmh, one thing I liked about this effort was possibility of stop bind-mounting
> >config files into the containers. I'd rather find a way to not need any
> >bindmount and have the services get their configs themselves.
>
> Probably sent too early!
>
> If we're not talking about OpenStack containers running in a COE, I guess this
> is fine. For k8s based deployments, I think I'd prefer having installers
> creating configmaps directly and use that. The reason is that depending on files
> that are in the host is not ideal for these scenarios. I hate this idea because
> it makes deployments inconsistent and I don't want that.
>
> Flavio
>

I'm not sure I understand how a configmap is any different from what is
proposed with confd in terms of deployment-specific data being added to
a container before it launches. Can you elaborate on that?

Doug

__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: [hidden email]?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [deployment] [oslo] [ansible] [tripleo] [kolla] [helm] Configuration management with etcd / confd

Steven Dake (stdake)
In reply to this post by Flavio Percoco-2
Flavio,

Atleast for the kubernetes variant of kolla, bindmounting will always be used as this is fundamentally how configmaps operate.  In order to maintain maximum flexilbility and compatibility with kubernetes, I am not keen to try a non-configmap way of doing things.

Regards
-steve

-----Original Message-----
From: Flavio Percoco <[hidden email]>
Reply-To: "OpenStack Development Mailing List (not for usage questions)" <[hidden email]>
Date: Thursday, June 8, 2017 at 9:23 AM
To: "OpenStack Development Mailing List (not for usage questions)" <[hidden email]>
Subject: Re: [openstack-dev] [deployment] [oslo] [ansible] [tripleo] [kolla] [helm] Configuration management with etcd / confd

    On 07/06/17 12:04 +0200, Bogdan Dobrelya wrote:
    >On 06.06.2017 18:08, Emilien Macchi wrote:
    >> Another benefit is that confd will generate a configuration file when
    >> the application will start. So if etcd is down *after* the app
    >> startup, it shouldn't break the service restart if we don't ask confd
    >> to re-generate the config. It's good for operators who were concerned
    >> about the fact the infrastructure would rely on etcd. In that case, we
    >> would only need etcd at the initial deployment (and during lifecycle
    >> actions like upgrades, etc).
    >>
    >> The downside is that in the case of containers, they would still have
    >> a configuration file within the container, and the whole goal of this
    >> feature was to externalize configuration data and stop having
    >> configuration files.
    >
    >It doesn't look a strict requirement. Those configs may (and should) be
    >bind-mounted into containers, as hostpath volumes. Or, am I missing
    >something what *does* make embedded configs a strict requirement?..
   
    mmh, one thing I liked about this effort was possibility of stop bind-mounting
    config files into the containers. I'd rather find a way to not need any
    bindmount and have the services get their configs themselves.
   
    Flavio
   
   
    --
    @flaper87
    Flavio Percoco
   

__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: [hidden email]?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [deployment] [oslo] [ansible] [tripleo] [kolla] [helm] Configuration management with etcd / confd

Steven Dake (stdake)
In reply to this post by Doug Hellmann-2
Doug,

In short, a configmap takes a bunch of config files, bundles them in a kubernetes object called a configmap, and then ships them to etcd.  When a pod is launched, the pod mounts the configmaps using tmpfs and the raw config files are available for use by the openstack services.

Operating on configmaps is much simpler and safer than using a different backing database for the configuration data.

Hope the information helps.

Ping me in #openstack-kolla if you have more questions.

Regards
-steve


-----Original Message-----
From: Doug Hellmann <[hidden email]>
Reply-To: "OpenStack Development Mailing List (not for usage questions)" <[hidden email]>
Date: Thursday, June 8, 2017 at 10:12 AM
To: openstack-dev <[hidden email]>
Subject: Re: [openstack-dev] [deployment] [oslo] [ansible] [tripleo] [kolla] [helm] Configuration management with etcd / confd

    Excerpts from Flavio Percoco's message of 2017-06-08 18:27:51 +0200:
    > On 08/06/17 18:23 +0200, Flavio Percoco wrote:
    > >On 07/06/17 12:04 +0200, Bogdan Dobrelya wrote:
    > >>On 06.06.2017 18:08, Emilien Macchi wrote:
    > >>>Another benefit is that confd will generate a configuration file when
    > >>>the application will start. So if etcd is down *after* the app
    > >>>startup, it shouldn't break the service restart if we don't ask confd
    > >>>to re-generate the config. It's good for operators who were concerned
    > >>>about the fact the infrastructure would rely on etcd. In that case, we
    > >>>would only need etcd at the initial deployment (and during lifecycle
    > >>>actions like upgrades, etc).
    > >>>
    > >>>The downside is that in the case of containers, they would still have
    > >>>a configuration file within the container, and the whole goal of this
    > >>>feature was to externalize configuration data and stop having
    > >>>configuration files.
    > >>
    > >>It doesn't look a strict requirement. Those configs may (and should) be
    > >>bind-mounted into containers, as hostpath volumes. Or, am I missing
    > >>something what *does* make embedded configs a strict requirement?..
    > >
    > >mmh, one thing I liked about this effort was possibility of stop bind-mounting
    > >config files into the containers. I'd rather find a way to not need any
    > >bindmount and have the services get their configs themselves.
    >
    > Probably sent too early!
    >
    > If we're not talking about OpenStack containers running in a COE, I guess this
    > is fine. For k8s based deployments, I think I'd prefer having installers
    > creating configmaps directly and use that. The reason is that depending on files
    > that are in the host is not ideal for these scenarios. I hate this idea because
    > it makes deployments inconsistent and I don't want that.
    >
    > Flavio
    >
   
    I'm not sure I understand how a configmap is any different from what is
    proposed with confd in terms of deployment-specific data being added to
    a container before it launches. Can you elaborate on that?
   
    Doug
   
    __________________________________________________________________________
    OpenStack Development Mailing List (not for usage questions)
    Unsubscribe: [hidden email]?subject:unsubscribe
    http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
   

__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: [hidden email]?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [deployment] [oslo] [ansible] [tripleo] [kolla] [helm] Configuration management with etcd / confd

Michał Jastrzębski
In reply to this post by Michał Jastrzębski
On 8 June 2017 at 09:50, Michał Jastrzębski <[hidden email]> wrote:

> On 8 June 2017 at 09:27, Flavio Percoco <[hidden email]> wrote:
>> On 08/06/17 18:23 +0200, Flavio Percoco wrote:
>>>
>>> On 07/06/17 12:04 +0200, Bogdan Dobrelya wrote:
>>>>
>>>> On 06.06.2017 18:08, Emilien Macchi wrote:
>>>>>
>>>>> Another benefit is that confd will generate a configuration file when
>>>>> the application will start. So if etcd is down *after* the app
>>>>> startup, it shouldn't break the service restart if we don't ask confd
>>>>> to re-generate the config. It's good for operators who were concerned
>>>>> about the fact the infrastructure would rely on etcd. In that case, we
>>>>> would only need etcd at the initial deployment (and during lifecycle
>>>>> actions like upgrades, etc).
>>>>>
>>>>> The downside is that in the case of containers, they would still have
>>>>> a configuration file within the container, and the whole goal of this
>>>>> feature was to externalize configuration data and stop having
>>>>> configuration files.
>>>>
>>>>
>>>> It doesn't look a strict requirement. Those configs may (and should) be
>>>> bind-mounted into containers, as hostpath volumes. Or, am I missing
>>>> something what *does* make embedded configs a strict requirement?..
>>>
>>>
>>> mmh, one thing I liked about this effort was possibility of stop
>>> bind-mounting
>>> config files into the containers. I'd rather find a way to not need any
>>> bindmount and have the services get their configs themselves.
>>
>>
>> Probably sent too early!
>>
>> If we're not talking about OpenStack containers running in a COE, I guess
>> this
>> is fine. For k8s based deployments, I think I'd prefer having installers
>> creating configmaps directly and use that. The reason is that depending on
>> files
>> that are in the host is not ideal for these scenarios. I hate this idea
>> because
>> it makes deployments inconsistent and I don't want that.
>
> Well, I disagree. If we're doing this we're essentially getting rid of
> "files" at all. It might actually be easier to handle from COE than
> configmap, as configmap has to be generated and when you get to host
> specific things it's quite a pain to handle. I, for one, would happily
> use cantral DB for config options if we define schema correctly.
>
> That being said defining schema correctly is quite a challenge. Few
> hard cases I see right now can be found in single use case - PCI
> Passthrough
>
> 1. I have multiple PCI devices in host, I need to specify list of them
> 2. PCI buses differes host to host, I need to specify groups of hosts
> that will share same bus configuration and reflect that in service
> config
>
> Maybe we should gather few of hard use cases like that and make sure
> we can address them in our config schema?

Speaking of hard use cases: here's another - config rolling upgrade +
config rollback. If we have single option in etcd, when service
restarts it automatically gets new config which creates funny edge
cases when you want to do rolling upgrade of config and some other
node fails->service restarts->config gets updated "accidentally".

>>
>> Flavio
>>
>> --
>> @flaper87
>> Flavio Percoco
>>
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: [hidden email]?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>

__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: [hidden email]?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [deployment] [oslo] [ansible] [tripleo] [kolla] [helm] Configuration management with etcd / confd

Doug Hellmann-2
In reply to this post by Steven Dake (stdake)
Excerpts from Steven Dake (stdake)'s message of 2017-06-08 17:51:48 +0000:
> Doug,
>
> In short, a configmap takes a bunch of config files, bundles them in a kubernetes object called a configmap, and then ships them to etcd.  When a pod is launched, the pod mounts the configmaps using tmpfs and the raw config files are available for use by the openstack services.

That sounds like what confd does. Something puts data into one of
several possible databases. confd takes it out and writes it to
file(s) when the container starts. The app in the container reads
the file(s).

It sounds like configmaps would work well, too, it just doesn't
sound like a fundamentally different solution.

Doug

>
> Operating on configmaps is much simpler and safer than using a different backing database for the configuration data.
>
> Hope the information helps.
>
> Ping me in #openstack-kolla if you have more questions.
>
> Regards
> -steve
>
> -----Original Message-----
> From: Doug Hellmann <[hidden email]>
> Reply-To: "OpenStack Development Mailing List (not for usage questions)" <[hidden email]>
> Date: Thursday, June 8, 2017 at 10:12 AM
> To: openstack-dev <[hidden email]>
> Subject: Re: [openstack-dev] [deployment] [oslo] [ansible] [tripleo] [kolla]    [helm] Configuration management with etcd / confd
>
>     Excerpts from Flavio Percoco's message of 2017-06-08 18:27:51 +0200:
>     > On 08/06/17 18:23 +0200, Flavio Percoco wrote:
>     > >On 07/06/17 12:04 +0200, Bogdan Dobrelya wrote:
>     > >>On 06.06.2017 18:08, Emilien Macchi wrote:
>     > >>>Another benefit is that confd will generate a configuration file when
>     > >>>the application will start. So if etcd is down *after* the app
>     > >>>startup, it shouldn't break the service restart if we don't ask confd
>     > >>>to re-generate the config. It's good for operators who were concerned
>     > >>>about the fact the infrastructure would rely on etcd. In that case, we
>     > >>>would only need etcd at the initial deployment (and during lifecycle
>     > >>>actions like upgrades, etc).
>     > >>>
>     > >>>The downside is that in the case of containers, they would still have
>     > >>>a configuration file within the container, and the whole goal of this
>     > >>>feature was to externalize configuration data and stop having
>     > >>>configuration files.
>     > >>
>     > >>It doesn't look a strict requirement. Those configs may (and should) be
>     > >>bind-mounted into containers, as hostpath volumes. Or, am I missing
>     > >>something what *does* make embedded configs a strict requirement?..
>     > >
>     > >mmh, one thing I liked about this effort was possibility of stop bind-mounting
>     > >config files into the containers. I'd rather find a way to not need any
>     > >bindmount and have the services get their configs themselves.
>     >
>     > Probably sent too early!
>     >
>     > If we're not talking about OpenStack containers running in a COE, I guess this
>     > is fine. For k8s based deployments, I think I'd prefer having installers
>     > creating configmaps directly and use that. The reason is that depending on files
>     > that are in the host is not ideal for these scenarios. I hate this idea because
>     > it makes deployments inconsistent and I don't want that.
>     >
>     > Flavio
>     >
>    
>     I'm not sure I understand how a configmap is any different from what is
>     proposed with confd in terms of deployment-specific data being added to
>     a container before it launches. Can you elaborate on that?
>    
>     Doug
>    
>     __________________________________________________________________________
>     OpenStack Development Mailing List (not for usage questions)
>     Unsubscribe: [hidden email]?subject:unsubscribe
>     http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>    
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: [hidden email]?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: [hidden email]?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [deployment] [oslo] [ansible] [tripleo] [kolla] [helm] Configuration management with etcd / confd

Fox, Kevin M
Yeah, I think k8s configmaps might be a good config mechanism for k8s based openstack deployment.

One feature that might help which is related to the etcd plugin would be a yaml/json plugin. It would allow more native looking configmaps.

Thanks,
Kevin
________________________________________
From: Doug Hellmann [[hidden email]]
Sent: Thursday, June 08, 2017 11:49 AM
To: openstack-dev
Subject: Re: [openstack-dev] [deployment] [oslo] [ansible] [tripleo] [kolla]    [helm] Configuration management with etcd / confd

Excerpts from Steven Dake (stdake)'s message of 2017-06-08 17:51:48 +0000:
> Doug,
>
> In short, a configmap takes a bunch of config files, bundles them in a kubernetes object called a configmap, and then ships them to etcd.  When a pod is launched, the pod mounts the configmaps using tmpfs and the raw config files are available for use by the openstack services.

That sounds like what confd does. Something puts data into one of
several possible databases. confd takes it out and writes it to
file(s) when the container starts. The app in the container reads
the file(s).

It sounds like configmaps would work well, too, it just doesn't
sound like a fundamentally different solution.

Doug

>
> Operating on configmaps is much simpler and safer than using a different backing database for the configuration data.
>
> Hope the information helps.
>
> Ping me in #openstack-kolla if you have more questions.
>
> Regards
> -steve
>
> -----Original Message-----
> From: Doug Hellmann <[hidden email]>
> Reply-To: "OpenStack Development Mailing List (not for usage questions)" <[hidden email]>
> Date: Thursday, June 8, 2017 at 10:12 AM
> To: openstack-dev <[hidden email]>
> Subject: Re: [openstack-dev] [deployment] [oslo] [ansible] [tripleo] [kolla]    [helm] Configuration management with etcd / confd
>
>     Excerpts from Flavio Percoco's message of 2017-06-08 18:27:51 +0200:
>     > On 08/06/17 18:23 +0200, Flavio Percoco wrote:
>     > >On 07/06/17 12:04 +0200, Bogdan Dobrelya wrote:
>     > >>On 06.06.2017 18:08, Emilien Macchi wrote:
>     > >>>Another benefit is that confd will generate a configuration file when
>     > >>>the application will start. So if etcd is down *after* the app
>     > >>>startup, it shouldn't break the service restart if we don't ask confd
>     > >>>to re-generate the config. It's good for operators who were concerned
>     > >>>about the fact the infrastructure would rely on etcd. In that case, we
>     > >>>would only need etcd at the initial deployment (and during lifecycle
>     > >>>actions like upgrades, etc).
>     > >>>
>     > >>>The downside is that in the case of containers, they would still have
>     > >>>a configuration file within the container, and the whole goal of this
>     > >>>feature was to externalize configuration data and stop having
>     > >>>configuration files.
>     > >>
>     > >>It doesn't look a strict requirement. Those configs may (and should) be
>     > >>bind-mounted into containers, as hostpath volumes. Or, am I missing
>     > >>something what *does* make embedded configs a strict requirement?..
>     > >
>     > >mmh, one thing I liked about this effort was possibility of stop bind-mounting
>     > >config files into the containers. I'd rather find a way to not need any
>     > >bindmount and have the services get their configs themselves.
>     >
>     > Probably sent too early!
>     >
>     > If we're not talking about OpenStack containers running in a COE, I guess this
>     > is fine. For k8s based deployments, I think I'd prefer having installers
>     > creating configmaps directly and use that. The reason is that depending on files
>     > that are in the host is not ideal for these scenarios. I hate this idea because
>     > it makes deployments inconsistent and I don't want that.
>     >
>     > Flavio
>     >
>
>     I'm not sure I understand how a configmap is any different from what is
>     proposed with confd in terms of deployment-specific data being added to
>     a container before it launches. Can you elaborate on that?
>
>     Doug
>
>     __________________________________________________________________________
>     OpenStack Development Mailing List (not for usage questions)
>     Unsubscribe: [hidden email]?subject:unsubscribe
>     http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: [hidden email]?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: [hidden email]?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: [hidden email]?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [deployment] [oslo] [ansible] [tripleo] [kolla] [helm] Configuration management with etcd / confd

Doug Hellmann-2
Excerpts from Fox, Kevin M's message of 2017-06-08 20:08:25 +0000:
> Yeah, I think k8s configmaps might be a good config mechanism for k8s based openstack deployment.
>
> One feature that might help which is related to the etcd plugin would be a yaml/json plugin. It would allow more native looking configmaps.

We have at least 2 mechanisms for getting config files into containers
without such significant changes to oslo.config.  At this point I'm
not sure it's necessary to do the driver work at all.

Doug

__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: [hidden email]?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [deployment] [oslo] [ansible] [tripleo] [kolla] [helm] Configuration management with etcd / confd

Emilien Macchi-4
In reply to this post by Doug Hellmann-2
On Thu, Jun 8, 2017 at 8:49 PM, Doug Hellmann <[hidden email]> wrote:

> Excerpts from Steven Dake (stdake)'s message of 2017-06-08 17:51:48 +0000:
>> Doug,
>>
>> In short, a configmap takes a bunch of config files, bundles them in a kubernetes object called a configmap, and then ships them to etcd.  When a pod is launched, the pod mounts the configmaps using tmpfs and the raw config files are available for use by the openstack services.
>
> That sounds like what confd does. Something puts data into one of
> several possible databases. confd takes it out and writes it to
> file(s) when the container starts. The app in the container reads
> the file(s).
>
> It sounds like configmaps would work well, too, it just doesn't
> sound like a fundamentally different solution.

Sorry for my lack of knowledge in ConfigMap but I'm trying to see how
we could bring pieces together.
Doug and I are currently investigating how oslo.config can be useful
to generate the parameters loaded by the application at startup,
without having to manage config with Puppet or Ansible.

If I understand correctly (and if not, please correct me, and maybe
propose something), we could use oslo.config to generate a portion of
ConfigMap (that can be imported in another ConfigMap iiuc) where we
would have parameters for one app.

Example with Keystone:

  apiVersion: v1
  kind: ConfigMap
  metadata:
    name: keystone-config
    namespace: DEFAULT
  data:
    debug: true
    rpc_backend: rabbit
    ... (parameters generated by oslo.config, and data fed by installers)

So iiuc we would give this file to k8s when deploying pods. Parameters
values would be automatically pushed into etcd, and used when
generating the configuration. Am I correct? (I need to understand if
we need to manually manage etcd key/values).

In that case, what deployments tools (like Kolla, TripleO, etc) would
expect from OpenStack to provide (tooling in oslo.config to generate
ConfigMap? etc.

Thanks for your help,

> Doug
>
>>
>> Operating on configmaps is much simpler and safer than using a different backing database for the configuration data.
>>
>> Hope the information helps.
>>
>> Ping me in #openstack-kolla if you have more questions.
>>
>> Regards
>> -steve
>>
>> -----Original Message-----
>> From: Doug Hellmann <[hidden email]>
>> Reply-To: "OpenStack Development Mailing List (not for usage questions)" <[hidden email]>
>> Date: Thursday, June 8, 2017 at 10:12 AM
>> To: openstack-dev <[hidden email]>
>> Subject: Re: [openstack-dev] [deployment] [oslo] [ansible] [tripleo] [kolla]    [helm] Configuration management with etcd / confd
>>
>>     Excerpts from Flavio Percoco's message of 2017-06-08 18:27:51 +0200:
>>     > On 08/06/17 18:23 +0200, Flavio Percoco wrote:
>>     > >On 07/06/17 12:04 +0200, Bogdan Dobrelya wrote:
>>     > >>On 06.06.2017 18:08, Emilien Macchi wrote:
>>     > >>>Another benefit is that confd will generate a configuration file when
>>     > >>>the application will start. So if etcd is down *after* the app
>>     > >>>startup, it shouldn't break the service restart if we don't ask confd
>>     > >>>to re-generate the config. It's good for operators who were concerned
>>     > >>>about the fact the infrastructure would rely on etcd. In that case, we
>>     > >>>would only need etcd at the initial deployment (and during lifecycle
>>     > >>>actions like upgrades, etc).
>>     > >>>
>>     > >>>The downside is that in the case of containers, they would still have
>>     > >>>a configuration file within the container, and the whole goal of this
>>     > >>>feature was to externalize configuration data and stop having
>>     > >>>configuration files.
>>     > >>
>>     > >>It doesn't look a strict requirement. Those configs may (and should) be
>>     > >>bind-mounted into containers, as hostpath volumes. Or, am I missing
>>     > >>something what *does* make embedded configs a strict requirement?..
>>     > >
>>     > >mmh, one thing I liked about this effort was possibility of stop bind-mounting
>>     > >config files into the containers. I'd rather find a way to not need any
>>     > >bindmount and have the services get their configs themselves.
>>     >
>>     > Probably sent too early!
>>     >
>>     > If we're not talking about OpenStack containers running in a COE, I guess this
>>     > is fine. For k8s based deployments, I think I'd prefer having installers
>>     > creating configmaps directly and use that. The reason is that depending on files
>>     > that are in the host is not ideal for these scenarios. I hate this idea because
>>     > it makes deployments inconsistent and I don't want that.
>>     >
>>     > Flavio
>>     >
>>
>>     I'm not sure I understand how a configmap is any different from what is
>>     proposed with confd in terms of deployment-specific data being added to
>>     a container before it launches. Can you elaborate on that?
>>
>>     Doug
>>
>>     __________________________________________________________________________
>>     OpenStack Development Mailing List (not for usage questions)
>>     Unsubscribe: [hidden email]?subject:unsubscribe
>>     http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: [hidden email]?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: [hidden email]?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



--
Emilien Macchi

__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: [hidden email]?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [deployment] [oslo] [ansible] [tripleo] [kolla] [helm] Configuration management with etcd / confd

Fox, Kevin M
In reply to this post by Doug Hellmann-2
That is possible. But, a yaml/json driver might still be good, regardless of the mechanism used to transfer the file.

So the driver abstraction still might be useful.

Thanks,
Kevin
________________________________________
From: Doug Hellmann [[hidden email]]
Sent: Thursday, June 08, 2017 1:19 PM
To: openstack-dev
Subject: Re: [openstack-dev] [deployment] [oslo] [ansible] [tripleo] [kolla]    [helm] Configuration management with etcd / confd

Excerpts from Fox, Kevin M's message of 2017-06-08 20:08:25 +0000:
> Yeah, I think k8s configmaps might be a good config mechanism for k8s based openstack deployment.
>
> One feature that might help which is related to the etcd plugin would be a yaml/json plugin. It would allow more native looking configmaps.

We have at least 2 mechanisms for getting config files into containers
without such significant changes to oslo.config.  At this point I'm
not sure it's necessary to do the driver work at all.

Doug

__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: [hidden email]?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: [hidden email]?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [deployment] [oslo] [ansible] [tripleo] [kolla] [helm] Configuration management with etcd / confd

Doug Hellmann-2
In reply to this post by Emilien Macchi-4
Excerpts from Emilien Macchi's message of 2017-06-08 22:20:34 +0200:

> On Thu, Jun 8, 2017 at 8:49 PM, Doug Hellmann <[hidden email]> wrote:
> > Excerpts from Steven Dake (stdake)'s message of 2017-06-08 17:51:48 +0000:
> >> Doug,
> >>
> >> In short, a configmap takes a bunch of config files, bundles them in a kubernetes object called a configmap, and then ships them to etcd.  When a pod is launched, the pod mounts the configmaps using tmpfs and the raw config files are available for use by the openstack services.
> >
> > That sounds like what confd does. Something puts data into one of
> > several possible databases. confd takes it out and writes it to
> > file(s) when the container starts. The app in the container reads
> > the file(s).
> >
> > It sounds like configmaps would work well, too, it just doesn't
> > sound like a fundamentally different solution.
>
> Sorry for my lack of knowledge in ConfigMap but I'm trying to see how
> we could bring pieces together.
> Doug and I are currently investigating how oslo.config can be useful
> to generate the parameters loaded by the application at startup,
> without having to manage config with Puppet or Ansible.
>
> If I understand correctly (and if not, please correct me, and maybe
> propose something), we could use oslo.config to generate a portion of
> ConfigMap (that can be imported in another ConfigMap iiuc) where we
> would have parameters for one app.
>
> Example with Keystone:
>
>   apiVersion: v1
>   kind: ConfigMap
>   metadata:
>     name: keystone-config
>     namespace: DEFAULT
>   data:
>     debug: true
>     rpc_backend: rabbit
>     ... (parameters generated by oslo.config, and data fed by installers)
>
> So iiuc we would give this file to k8s when deploying pods. Parameters
> values would be automatically pushed into etcd, and used when
> generating the configuration. Am I correct? (I need to understand if
> we need to manually manage etcd key/values).
>
> In that case, what deployments tools (like Kolla, TripleO, etc) would
> expect from OpenStack to provide (tooling in oslo.config to generate
> ConfigMap? etc.
>
> Thanks for your help,

Based on [1] I think the idea is to write the entire config file
for the service outside of the container, upload it to the configmap,
then configure the pod to create a volume and write the configmap
contents to the volume before launching the container. It's sort of like
nova's file-injection feature.

The approach seems appealing, although I don't fully understand the
issues others have raised with adding volumes to containers.

Doug

[1] https://kubernetes.io/docs/tasks/configure-pod-container/configure-pod-configmap/#populate-a-volume-with-data-stored-in-a-configmap

__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: [hidden email]?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [deployment] [oslo] [ansible] [tripleo] [kolla] [helm] Configuration management with etcd / confd

Fox, Kevin M
In reply to this post by Emilien Macchi-4
There are two issues conflated here maybe?

The first is a mechanism to use oslo.config to dump out example settings that could be loaded into a reference ConfigMap or etcd or something. I think there is a PS up for that.

The other is a way to get the data back into oslo.config.

etcd is one way.
using a ConfigMap to ship a file into a container to be read by oslo.config with a json/yaml/ini file driver is another.

Thanks,
Kevin
________________________________________
From: Emilien Macchi [[hidden email]]
Sent: Thursday, June 08, 2017 1:20 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [deployment] [oslo] [ansible] [tripleo] [kolla] [helm] Configuration management with etcd / confd

On Thu, Jun 8, 2017 at 8:49 PM, Doug Hellmann <[hidden email]> wrote:

> Excerpts from Steven Dake (stdake)'s message of 2017-06-08 17:51:48 +0000:
>> Doug,
>>
>> In short, a configmap takes a bunch of config files, bundles them in a kubernetes object called a configmap, and then ships them to etcd.  When a pod is launched, the pod mounts the configmaps using tmpfs and the raw config files are available for use by the openstack services.
>
> That sounds like what confd does. Something puts data into one of
> several possible databases. confd takes it out and writes it to
> file(s) when the container starts. The app in the container reads
> the file(s).
>
> It sounds like configmaps would work well, too, it just doesn't
> sound like a fundamentally different solution.

Sorry for my lack of knowledge in ConfigMap but I'm trying to see how
we could bring pieces together.
Doug and I are currently investigating how oslo.config can be useful
to generate the parameters loaded by the application at startup,
without having to manage config with Puppet or Ansible.

If I understand correctly (and if not, please correct me, and maybe
propose something), we could use oslo.config to generate a portion of
ConfigMap (that can be imported in another ConfigMap iiuc) where we
would have parameters for one app.

Example with Keystone:

  apiVersion: v1
  kind: ConfigMap
  metadata:
    name: keystone-config
    namespace: DEFAULT
  data:
    debug: true
    rpc_backend: rabbit
    ... (parameters generated by oslo.config, and data fed by installers)

So iiuc we would give this file to k8s when deploying pods. Parameters
values would be automatically pushed into etcd, and used when
generating the configuration. Am I correct? (I need to understand if
we need to manually manage etcd key/values).

In that case, what deployments tools (like Kolla, TripleO, etc) would
expect from OpenStack to provide (tooling in oslo.config to generate
ConfigMap? etc.

Thanks for your help,

> Doug
>
>>
>> Operating on configmaps is much simpler and safer than using a different backing database for the configuration data.
>>
>> Hope the information helps.
>>
>> Ping me in #openstack-kolla if you have more questions.
>>
>> Regards
>> -steve
>>
>> -----Original Message-----
>> From: Doug Hellmann <[hidden email]>
>> Reply-To: "OpenStack Development Mailing List (not for usage questions)" <[hidden email]>
>> Date: Thursday, June 8, 2017 at 10:12 AM
>> To: openstack-dev <[hidden email]>
>> Subject: Re: [openstack-dev] [deployment] [oslo] [ansible] [tripleo] [kolla]    [helm] Configuration management with etcd / confd
>>
>>     Excerpts from Flavio Percoco's message of 2017-06-08 18:27:51 +0200:
>>     > On 08/06/17 18:23 +0200, Flavio Percoco wrote:
>>     > >On 07/06/17 12:04 +0200, Bogdan Dobrelya wrote:
>>     > >>On 06.06.2017 18:08, Emilien Macchi wrote:
>>     > >>>Another benefit is that confd will generate a configuration file when
>>     > >>>the application will start. So if etcd is down *after* the app
>>     > >>>startup, it shouldn't break the service restart if we don't ask confd
>>     > >>>to re-generate the config. It's good for operators who were concerned
>>     > >>>about the fact the infrastructure would rely on etcd. In that case, we
>>     > >>>would only need etcd at the initial deployment (and during lifecycle
>>     > >>>actions like upgrades, etc).
>>     > >>>
>>     > >>>The downside is that in the case of containers, they would still have
>>     > >>>a configuration file within the container, and the whole goal of this
>>     > >>>feature was to externalize configuration data and stop having
>>     > >>>configuration files.
>>     > >>
>>     > >>It doesn't look a strict requirement. Those configs may (and should) be
>>     > >>bind-mounted into containers, as hostpath volumes. Or, am I missing
>>     > >>something what *does* make embedded configs a strict requirement?..
>>     > >
>>     > >mmh, one thing I liked about this effort was possibility of stop bind-mounting
>>     > >config files into the containers. I'd rather find a way to not need any
>>     > >bindmount and have the services get their configs themselves.
>>     >
>>     > Probably sent too early!
>>     >
>>     > If we're not talking about OpenStack containers running in a COE, I guess this
>>     > is fine. For k8s based deployments, I think I'd prefer having installers
>>     > creating configmaps directly and use that. The reason is that depending on files
>>     > that are in the host is not ideal for these scenarios. I hate this idea because
>>     > it makes deployments inconsistent and I don't want that.
>>     >
>>     > Flavio
>>     >
>>
>>     I'm not sure I understand how a configmap is any different from what is
>>     proposed with confd in terms of deployment-specific data being added to
>>     a container before it launches. Can you elaborate on that?
>>
>>     Doug
>>
>>     __________________________________________________________________________
>>     OpenStack Development Mailing List (not for usage questions)
>>     Unsubscribe: [hidden email]?subject:unsubscribe
>>     http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: [hidden email]?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: [hidden email]?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



--
Emilien Macchi

__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: [hidden email]?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: [hidden email]?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
12
Loading...