[TripleO] PSA lets use deploy_steps_tasks

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

[TripleO] PSA lets use deploy_steps_tasks

Dan Prince
I pushed a patch[1] to update our containerized deployment
architecture docs yesterday. There are 2 new fairly useful sections we
can leverage with TripleO's stepwise deployment. They appear to be
used somewhat sparingly so I wanted to get the word out.

The first is 'deploy_steps_tasks' which gives you a means to run
Ansible snippets on each node/role in a stepwise fashion during
deployment. Previously it was only possible to execute puppet or
docker commands where as now that we have deploy_steps_tasks we can
execute ad-hoc ansible in the same manner.

The second is 'external_deploy_tasks' which allows you to use run
Ansible snippets on the Undercloud during stepwise deployment. This is
probably most useful for driving an external installer but might also
help with some complex tasks that need to originate from a single
Ansible client.

The only downside I see to these approaches is that both appear to be
implemented with Ansible's default linear strategy. I saw shardy's
comment here [2] that the :free strategy does not yet apparently work
with the any_errors_fatal option. Perhaps we can reach out to someone
in the Ansible community in this regard to improve running these
things in parallel like TripleO used to work with Heat agents.

This is also how host_prep_tasks is implemented which BTW we should
now get rid of as a duplicate architectural step since we have
deploy_steps_tasks anyway.

[1] https://review.openstack.org/#/c/614822/
[2] http://git.openstack.org/cgit/openstack/tripleo-heat-templates/tree/common/deploy-steps.j2#n554

__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: [hidden email]?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Reply | Threaded
Open this post in threaded view
|

Re: [TripleO] PSA lets use deploy_steps_tasks

Juan Antonio Osorio Robles
Thanks! We have been slow to update our docs. I did put up a blog post
about these sections of the templates [1], in case folks find that useful.


[1] http://jaormx.github.io/2018/dissecting-tripleo-service-templates-p2/

On 11/2/18 3:39 PM, Dan Prince wrote:

> I pushed a patch[1] to update our containerized deployment
> architecture docs yesterday. There are 2 new fairly useful sections we
> can leverage with TripleO's stepwise deployment. They appear to be
> used somewhat sparingly so I wanted to get the word out.
>
> The first is 'deploy_steps_tasks' which gives you a means to run
> Ansible snippets on each node/role in a stepwise fashion during
> deployment. Previously it was only possible to execute puppet or
> docker commands where as now that we have deploy_steps_tasks we can
> execute ad-hoc ansible in the same manner.
>
> The second is 'external_deploy_tasks' which allows you to use run
> Ansible snippets on the Undercloud during stepwise deployment. This is
> probably most useful for driving an external installer but might also
> help with some complex tasks that need to originate from a single
> Ansible client.
>
> The only downside I see to these approaches is that both appear to be
> implemented with Ansible's default linear strategy. I saw shardy's
> comment here [2] that the :free strategy does not yet apparently work
> with the any_errors_fatal option. Perhaps we can reach out to someone
> in the Ansible community in this regard to improve running these
> things in parallel like TripleO used to work with Heat agents.
>
> This is also how host_prep_tasks is implemented which BTW we should
> now get rid of as a duplicate architectural step since we have
> deploy_steps_tasks anyway.
>
> [1] https://review.openstack.org/#/c/614822/
> [2] http://git.openstack.org/cgit/openstack/tripleo-heat-templates/tree/common/deploy-steps.j2#n554
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: [hidden email]?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: [hidden email]?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Reply | Threaded
Open this post in threaded view
|

Re: [TripleO] PSA lets use deploy_steps_tasks

James Slagle
In reply to this post by Dan Prince
On Fri, Nov 2, 2018 at 9:39 AM Dan Prince <[hidden email]> wrote:

>
> I pushed a patch[1] to update our containerized deployment
> architecture docs yesterday. There are 2 new fairly useful sections we
> can leverage with TripleO's stepwise deployment. They appear to be
> used somewhat sparingly so I wanted to get the word out.
>
> The first is 'deploy_steps_tasks' which gives you a means to run
> Ansible snippets on each node/role in a stepwise fashion during
> deployment. Previously it was only possible to execute puppet or
> docker commands where as now that we have deploy_steps_tasks we can
> execute ad-hoc ansible in the same manner.
>
> The second is 'external_deploy_tasks' which allows you to use run
> Ansible snippets on the Undercloud during stepwise deployment. This is
> probably most useful for driving an external installer but might also
> help with some complex tasks that need to originate from a single
> Ansible client.

+1


> The only downside I see to these approaches is that both appear to be
> implemented with Ansible's default linear strategy. I saw shardy's
> comment here [2] that the :free strategy does not yet apparently work
> with the any_errors_fatal option. Perhaps we can reach out to someone
> in the Ansible community in this regard to improve running these
> things in parallel like TripleO used to work with Heat agents.

It's effectively parallel across one role at a time at the moment, up
to the number of configured forks (default: 25). The reason it won't
parallelize across roles, is because it's a different task file used
with import_tasks for each role. Ansible won't run that in parallel
since the task list is different.

I was able to make this parallel across roles for the pre and post
deployments by making the task file the same for each role, and
controlling the difference with group and host vars:
https://review.openstack.org/#/c/574474/
From Ansible's perspective, the task list is now the same for each
host, although different things will be done depending on the value of
vars for each host.

It's possible a similar approach could be done with the other
interfaces you point out here.

In addition to the any_errors_fatal issue when using strategy:free, is
that you'd also lose the grouping of the task output per role after
each task finishes. This is mostly cosmetic, but using free does
create a lot more noisier output IMO.

--
-- James Slagle
--

__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: [hidden email]?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Reply | Threaded
Open this post in threaded view
|

Re: [TripleO] PSA lets use deploy_steps_tasks

Cédric Jeanneret-2
In reply to this post by Dan Prince
On 11/2/18 2:39 PM, Dan Prince wrote:
> I pushed a patch[1] to update our containerized deployment
> architecture docs yesterday. There are 2 new fairly useful sections we
> can leverage with TripleO's stepwise deployment. They appear to be
> used somewhat sparingly so I wanted to get the word out.

Good thing, it's important to highlight this feature and explain how it
works, big thumb up Dan!

>
> The first is 'deploy_steps_tasks' which gives you a means to run
> Ansible snippets on each node/role in a stepwise fashion during
> deployment. Previously it was only possible to execute puppet or
> docker commands where as now that we have deploy_steps_tasks we can
> execute ad-hoc ansible in the same manner.

I'm wondering if such a thing could be used for the "inflight
validations" - i.e. a step to validate a service/container is working as
expected once it's deployed, in order to get early failure.
For instance, we deploy a rabbitmq container, and right after it's
deployed, we'd like to ensure it's actually running and works as
expected before going forward in the deploy.

Care to have a look at that spec[1] and see if, instead of adding a new
"validation_tasks" entry, we could "just" use the "deploy_steps_tasks"
with the right step number? That would be really, really cool, and will
probably avoid a lot of code in the end :).

Thank you!

C.

[1] https://review.openstack.org/#/c/602007/

>
> The second is 'external_deploy_tasks' which allows you to use run
> Ansible snippets on the Undercloud during stepwise deployment. This is
> probably most useful for driving an external installer but might also
> help with some complex tasks that need to originate from a single
> Ansible client.
>
> The only downside I see to these approaches is that both appear to be
> implemented with Ansible's default linear strategy. I saw shardy's
> comment here [2] that the :free strategy does not yet apparently work
> with the any_errors_fatal option. Perhaps we can reach out to someone
> in the Ansible community in this regard to improve running these
> things in parallel like TripleO used to work with Heat agents.
>
> This is also how host_prep_tasks is implemented which BTW we should
> now get rid of as a duplicate architectural step since we have
> deploy_steps_tasks anyway.
>
> [1] https://review.openstack.org/#/c/614822/
> [2] http://git.openstack.org/cgit/openstack/tripleo-heat-templates/tree/common/deploy-steps.j2#n554
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: [hidden email]?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
--
Cédric Jeanneret
Software Engineer
DFG:DF


__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: [hidden email]?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: [TripleO] PSA lets use deploy_steps_tasks

Dan Prince
On Mon, Nov 5, 2018 at 4:06 AM Cédric Jeanneret <[hidden email]> wrote:

>
> On 11/2/18 2:39 PM, Dan Prince wrote:
> > I pushed a patch[1] to update our containerized deployment
> > architecture docs yesterday. There are 2 new fairly useful sections we
> > can leverage with TripleO's stepwise deployment. They appear to be
> > used somewhat sparingly so I wanted to get the word out.
>
> Good thing, it's important to highlight this feature and explain how it
> works, big thumb up Dan!
>
> >
> > The first is 'deploy_steps_tasks' which gives you a means to run
> > Ansible snippets on each node/role in a stepwise fashion during
> > deployment. Previously it was only possible to execute puppet or
> > docker commands where as now that we have deploy_steps_tasks we can
> > execute ad-hoc ansible in the same manner.
>
> I'm wondering if such a thing could be used for the "inflight
> validations" - i.e. a step to validate a service/container is working as
> expected once it's deployed, in order to get early failure.
> For instance, we deploy a rabbitmq container, and right after it's
> deployed, we'd like to ensure it's actually running and works as
> expected before going forward in the deploy.
>
> Care to have a look at that spec[1] and see if, instead of adding a new
> "validation_tasks" entry, we could "just" use the "deploy_steps_tasks"
> with the right step number? That would be really, really cool, and will
> probably avoid a lot of code in the end :).

It could work fine I think. As deploy-steps-tasks runs before the
"common container/baremetal" actions special care would need to be
taken so that validations for a containers startup occur at the
beginning of the next step. So a container started at step 2 would be
validated early in step 3. This may also require us to have a "post"
deploy_steps_tasks" iteration so that we can validate late starting
containers.

If if we use the more generic deploy_steps_tasks section we'd probably
rely on conventions to always add Ansible tags onto the validation
tasks. These could be useful for those wanting to selectively execute
them externally (not sure if that was part of your spec but I could
see someone wanting this).

Dan

>
> Thank you!
>
> C.
>
> [1] https://review.openstack.org/#/c/602007/
>
> >
> > The second is 'external_deploy_tasks' which allows you to use run
> > Ansible snippets on the Undercloud during stepwise deployment. This is
> > probably most useful for driving an external installer but might also
> > help with some complex tasks that need to originate from a single
> > Ansible client.
> >
> > The only downside I see to these approaches is that both appear to be
> > implemented with Ansible's default linear strategy. I saw shardy's
> > comment here [2] that the :free strategy does not yet apparently work
> > with the any_errors_fatal option. Perhaps we can reach out to someone
> > in the Ansible community in this regard to improve running these
> > things in parallel like TripleO used to work with Heat agents.
> >
> > This is also how host_prep_tasks is implemented which BTW we should
> > now get rid of as a duplicate architectural step since we have
> > deploy_steps_tasks anyway.
> >
> > [1] https://review.openstack.org/#/c/614822/
> > [2] http://git.openstack.org/cgit/openstack/tripleo-heat-templates/tree/common/deploy-steps.j2#n554
> >
> > __________________________________________________________________________
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe: [hidden email]?subject:unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
>
> --
> Cédric Jeanneret
> Software Engineer
> DFG:DF
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: [hidden email]?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: [hidden email]?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev