[TripleO][Heat] Conditionally passing properties in Heat

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[TripleO][Heat] Conditionally passing properties in Heat

Dan Sneddon
I'm implementing predictable control plane IPs for spine/leaf, and I'm
running into a problem implementing this in the TripleO Heat templates.

I have a review in progress [1] that works, but fails on upgrade, so I'm
looking for an alternative approach. I'm trying to influence the IP
address that is selected for overcloud nodes' Control Plane IP. Here is
the current construct:

  Controller:
    type: OS::TripleO::Server
    metadata:
      os-collect-config:
        command: {get_param: ConfigCommand}
    properties:
      image: {get_param: controllerImage}
      image_update_policy: {get_param: ImageUpdatePolicy}
      flavor: {get_param: OvercloudControlFlavor}
      key_name: {get_param: KeyName}
      networks:
        - network: ctlplane  # <- Here's where the port is created

If I add fixed_ip: to the networks element at the end of the above, I
can select an IP address from the 'ctlplane' network, like this:

      networks:
        - network: ctlplane
          fixed_ip: {get_attr: [ControlPlanePort, ip_address]}

But the problem is that if I pass a blank string to fixed_ip, I get an
error on deployment. This means that the old behavior of automatically
selecting an IP doesn't work.

I thought I has solved this by passing an external Neutron port, like this:

      networks:
        - network: ctlplane
          port: {get_attr: [ControlPlanePort, port_id]}

Which works for deployments, but that fails on upgrades, since the
original port was created as part of the Nova::Server resource, instead
of being an external resource.

I'm now looking for a way to use Heat conditionals to apply the fixed_ip
only if the value is not unset. Looking at the intrinsic functions [2],
I don't see a way to do this. Is what I'm trying to do with Heat possible?

Another option I'm exploring is conditionally applying resources. It
appears that would require duplicating the entire TripleO::Server stanza
in *-role.yaml so that there is one that uses fixed_ip and one that does
not. Which one is applied would be based on a condition that tested
whether fixed_ip was blank or not. The downside of that is that it would
make the role definition confusing because there would be a large
resource that was implemented twice, with only one line difference
between them.

Does anyone have any ideas how to go about this?

[1] - https://review.openstack.org/#/c/413278/
[2] -
https://docs.openstack.org/developer/heat/template_guide/hot_spec.html#intrinsic-functions

--
Dan Sneddon         |  Senior Principal Software Engineer
[hidden email] |  redhat.com/openstack
dsneddon:irc        |  @dxs:twitter

__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: [hidden email]?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [TripleO][Heat] Conditionally passing properties in Heat

Thomas Herve-2
On Wed, Apr 12, 2017 at 9:00 PM, Dan Sneddon <[hidden email]> wrote:

> I'm implementing predictable control plane IPs for spine/leaf, and I'm
> running into a problem implementing this in the TripleO Heat templates.
>
> I have a review in progress [1] that works, but fails on upgrade, so I'm
> looking for an alternative approach. I'm trying to influence the IP
> address that is selected for overcloud nodes' Control Plane IP. Here is
> the current construct:
>
>   Controller:
>     type: OS::TripleO::Server
>     metadata:
>       os-collect-config:
>         command: {get_param: ConfigCommand}
>     properties:
>       image: {get_param: controllerImage}
>       image_update_policy: {get_param: ImageUpdatePolicy}
>       flavor: {get_param: OvercloudControlFlavor}
>       key_name: {get_param: KeyName}
>       networks:
>         - network: ctlplane  # <- Here's where the port is created
>
> If I add fixed_ip: to the networks element at the end of the above, I
> can select an IP address from the 'ctlplane' network, like this:
>
>       networks:
>         - network: ctlplane
>           fixed_ip: {get_attr: [ControlPlanePort, ip_address]}
>
> But the problem is that if I pass a blank string to fixed_ip, I get an
> error on deployment. This means that the old behavior of automatically
> selecting an IP doesn't work.
>
> I thought I has solved this by passing an external Neutron port, like this:
>
>       networks:
>         - network: ctlplane
>           port: {get_attr: [ControlPlanePort, port_id]}
>
> Which works for deployments, but that fails on upgrades, since the
> original port was created as part of the Nova::Server resource, instead
> of being an external resource.

Can you detail how it fails? I was under the impression we never
replaced servers no matter what (or we try to do that, at least). Is
the issue that your new port is not the correct one?

> I'm now looking for a way to use Heat conditionals to apply the fixed_ip
> only if the value is not unset. Looking at the intrinsic functions [2],
> I don't see a way to do this. Is what I'm trying to do with Heat possible?

You should be able to write something like that (not tested):

networks:
  if:
    - <my condition>
    - network: ctlplane
      fixed_ip: {get_attr: [ControlPlanePort, ip_address]}
    - network: ctlplane

The question is how to define your condition. Maybe:

conditions:
  fixed_ip_condition:
     not:
        equals:
          - {get_attr: [ControlPlanePort, ip_address]}
          - ''

To get back to the problem you stated first.


> Another option I'm exploring is conditionally applying resources. It
> appears that would require duplicating the entire TripleO::Server stanza
> in *-role.yaml so that there is one that uses fixed_ip and one that does
> not. Which one is applied would be based on a condition that tested
> whether fixed_ip was blank or not. The downside of that is that it would
> make the role definition confusing because there would be a large
> resource that was implemented twice, with only one line difference
> between them.

You can define properties with conditions, so you shouldn't need to
rewrite everything.

--
Thomas

__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: [hidden email]?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [TripleO][Heat] Conditionally passing properties in Heat

Dan Sneddon
On 04/12/2017 01:22 PM, Thomas Herve wrote:

> On Wed, Apr 12, 2017 at 9:00 PM, Dan Sneddon <[hidden email]> wrote:
>> I'm implementing predictable control plane IPs for spine/leaf, and I'm
>> running into a problem implementing this in the TripleO Heat templates.
>>
>> I have a review in progress [1] that works, but fails on upgrade, so I'm
>> looking for an alternative approach. I'm trying to influence the IP
>> address that is selected for overcloud nodes' Control Plane IP. Here is
>> the current construct:
>>
>>   Controller:
>>     type: OS::TripleO::Server
>>     metadata:
>>       os-collect-config:
>>         command: {get_param: ConfigCommand}
>>     properties:
>>       image: {get_param: controllerImage}
>>       image_update_policy: {get_param: ImageUpdatePolicy}
>>       flavor: {get_param: OvercloudControlFlavor}
>>       key_name: {get_param: KeyName}
>>       networks:
>>         - network: ctlplane  # <- Here's where the port is created
>>
>> If I add fixed_ip: to the networks element at the end of the above, I
>> can select an IP address from the 'ctlplane' network, like this:
>>
>>       networks:
>>         - network: ctlplane
>>           fixed_ip: {get_attr: [ControlPlanePort, ip_address]}
>>
>> But the problem is that if I pass a blank string to fixed_ip, I get an
>> error on deployment. This means that the old behavior of automatically
>> selecting an IP doesn't work.
>>
>> I thought I has solved this by passing an external Neutron port, like this:
>>
>>       networks:
>>         - network: ctlplane
>>           port: {get_attr: [ControlPlanePort, port_id]}
>>
>> Which works for deployments, but that fails on upgrades, since the
>> original port was created as part of the Nova::Server resource, instead
>> of being an external resource.
>
> Can you detail how it fails? I was under the impression we never
> replaced servers no matter what (or we try to do that, at least). Is
> the issue that your new port is not the correct one?
>
>> I'm now looking for a way to use Heat conditionals to apply the fixed_ip
>> only if the value is not unset. Looking at the intrinsic functions [2],
>> I don't see a way to do this. Is what I'm trying to do with Heat possible?
>
> You should be able to write something like that (not tested):
>
> networks:
>   if:
>     - <my condition>
>     - network: ctlplane
>       fixed_ip: {get_attr: [ControlPlanePort, ip_address]}
>     - network: ctlplane
>
> The question is how to define your condition. Maybe:
>
> conditions:
>   fixed_ip_condition:
>      not:
>         equals:
>           - {get_attr: [ControlPlanePort, ip_address]}
>           - ''
>
> To get back to the problem you stated first.
>
>
>> Another option I'm exploring is conditionally applying resources. It
>> appears that would require duplicating the entire TripleO::Server stanza
>> in *-role.yaml so that there is one that uses fixed_ip and one that does
>> not. Which one is applied would be based on a condition that tested
>> whether fixed_ip was blank or not. The downside of that is that it would
>> make the role definition confusing because there would be a large
>> resource that was implemented twice, with only one line difference
>> between them.
>
> You can define properties with conditions, so you shouldn't need to
> rewrite everything.
>

Thomas,

Thanks, I will try your suggestions and that should get me closer.

The full error log is available here:
http://logs.openstack.org/78/413278/11/check-tripleo/gate-tripleo-ci-centos-7-ovb-updates/8d91762/console.html

Here are the errors I am getting:

2017-04-12 00:26:34.436655 | 2017-04-12 00:26:29Z
[overcloud-CephStorage-bkucn6ign34i-0-2yq2jbtwuu7k.CephStorage]:
UPDATE_FAILED  RetryError: resources.CephStorage: RetryError[<Future at
0xdd62550 state=finished returned bool>]
2017-04-12 00:26:34.436808 | 2017-04-12 00:26:29Z
[overcloud-CephStorage-bkucn6ign34i-0-2yq2jbtwuu7k]: UPDATE_FAILED
RetryError: resources.CephStorage: RetryError[<Future at 0xdd62550
state=finished returned bool>]
2017-04-12 00:26:34.436903 | 2017-04-12 00:26:29Z
[overcloud-CephStorage-bkucn6ign34i.0]: UPDATE_FAILED  resources[0]:
RetryError: resources.CephStorage: RetryError[<Future at 0xdd62550
state=finished returned bool>]
2017-04-12 00:26:34.436989 | 2017-04-12 00:26:29Z
[overcloud-CephStorage-bkucn6ign34i]: UPDATE_FAILED  resources[0]:
RetryError: resources.CephStorage: RetryError[<Future at 0xdd62550
state=finished returned bool>]
2017-04-12 00:26:34.437078 | 2017-04-12 00:26:30Z
[overcloud-Controller-3lf3jauv4cbc-0-ydowkb3nwsso.Controller]:
UPDATE_FAILED  RetryError: resources.Controller: RetryError[<Future at
0xdc79b50 state=finished returned bool>]
2017-04-12 00:26:34.437173 | 2017-04-12 00:26:30Z
[overcloud-Controller-3lf3jauv4cbc-0-ydowkb3nwsso]: UPDATE_FAILED
RetryError: resources.Controller: RetryError[<Future at 0xdc79b50
state=finished returned bool>]
2017-04-12 00:26:34.437269 | 2017-04-12 00:26:30Z [CephStorage]:
UPDATE_FAILED  resources.CephStorage: resources[0]: RetryError:
resources.CephStorage: RetryError[<Future at 0xdd62550 state=finished
returned bool>]

--
Dan Sneddon         |  Senior Principal Software Engineer
[hidden email] |  redhat.com/openstack
dsneddon:irc        |  @dxs:twitter

__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: [hidden email]?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [TripleO][Heat] Conditionally passing properties in Heat

Rabi Mishra
On Thu, Apr 13, 2017 at 2:14 AM, Dan Sneddon <[hidden email]> wrote:
On 04/12/2017 01:22 PM, Thomas Herve wrote:
> On Wed, Apr 12, 2017 at 9:00 PM, Dan Sneddon <[hidden email]> wrote:
>> I'm implementing predictable control plane IPs for spine/leaf, and I'm
>> running into a problem implementing this in the TripleO Heat templates.
>>
>> I have a review in progress [1] that works, but fails on upgrade, so I'm
>> looking for an alternative approach. I'm trying to influence the IP
>> address that is selected for overcloud nodes' Control Plane IP. Here is
>> the current construct:
>>
>>   Controller:
>>     type: OS::TripleO::Server
>>     metadata:
>>       os-collect-config:
>>         command: {get_param: ConfigCommand}
>>     properties:
>>       image: {get_param: controllerImage}
>>       image_update_policy: {get_param: ImageUpdatePolicy}
>>       flavor: {get_param: OvercloudControlFlavor}
>>       key_name: {get_param: KeyName}
>>       networks:
>>         - network: ctlplane  # <- Here's where the port is created
>>
>> If I add fixed_ip: to the networks element at the end of the above, I
>> can select an IP address from the 'ctlplane' network, like this:
>>
>>       networks:
>>         - network: ctlplane
>>           fixed_ip: {get_attr: [ControlPlanePort, ip_address]}
>>
>> But the problem is that if I pass a blank string to fixed_ip, I get an
>> error on deployment. This means that the old behavior of automatically
>> selecting an IP doesn't work.
>>
>> I thought I has solved this by passing an external Neutron port, like this:
>>
>>       networks:
>>         - network: ctlplane
>>           port: {get_attr: [ControlPlanePort, port_id]}
>>
>> Which works for deployments, but that fails on upgrades, since the
>> original port was created as part of the Nova::Server resource, instead
>> of being an external resource.
>
> Can you detail how it fails? I was under the impression we never
> replaced servers no matter what (or we try to do that, at least). Is
> the issue that your new port is not the correct one?
>
>> I'm now looking for a way to use Heat conditionals to apply the fixed_ip
>> only if the value is not unset. Looking at the intrinsic functions [2],
>> I don't see a way to do this. Is what I'm trying to do with Heat possible?
>
> You should be able to write something like that (not tested):
>
> networks:
>   if:
>     - <my condition>
>     - network: ctlplane
>       fixed_ip: {get_attr: [ControlPlanePort, ip_address]}
>     - network: ctlplane
>
> The question is how to define your condition. Maybe:
>
> conditions:
>   fixed_ip_condition:
>      not:
>         equals:
>           - {get_attr: [ControlPlanePort, ip_address]}
>           - ''
>
> To get back to the problem you stated first.
>
>
>> Another option I'm exploring is conditionally applying resources. It
>> appears that would require duplicating the entire TripleO::Server stanza
>> in *-role.yaml so that there is one that uses fixed_ip and one that does
>> not. Which one is applied would be based on a condition that tested
>> whether fixed_ip was blank or not. The downside of that is that it would
>> make the role definition confusing because there would be a large
>> resource that was implemented twice, with only one line difference
>> between them.
>
> You can define properties with conditions, so you shouldn't need to
> rewrite everything.
>

Thomas,

Thanks, I will try your suggestions and that should get me closer.

The full error log is available here:
http://logs.openstack.org/78/413278/11/check-tripleo/gate-tripleo-ci-centos-7-ovb-updates/8d91762/console.html

We do an interface_detach/attach when a port is replaced.
It seems to be failing[1] as this is not implemented for ironic/baremetal driver.  I could see a patch[2] to add that functionality though.

[1] http://logs.openstack.org/78/413278/11/check-tripleo/gate-tripleo-ci-centos-7-ovb-updates/8d91762/logs/undercloud/var/log/nova/nova-compute.txt.gz#_2017-04-12_00_26_15_475

[2] https://review.openstack.org/#/c/419975/

We retry a few times to check whether the detach/attach is complete(it's an async operation in nova and takes time), so the cryptic error below is coming from tenacity library which fails after the configured number of attempts.

Here are the errors I am getting:

2017-04-12 00:26:34.436655 | 2017-04-12 00:26:29Z
[overcloud-CephStorage-bkucn6ign34i-0-2yq2jbtwuu7k.CephStorage]:
UPDATE_FAILED  RetryError: resources.CephStorage: RetryError[<Future at
0xdd62550 state=finished returned bool>]
2017-04-12 00:26:34.436808 | 2017-04-12 00:26:29Z
[overcloud-CephStorage-bkucn6ign34i-0-2yq2jbtwuu7k]: UPDATE_FAILED
RetryError: resources.CephStorage: RetryError[<Future at 0xdd62550
state=finished returned bool>]
2017-04-12 00:26:34.436903 | 2017-04-12 00:26:29Z
[overcloud-CephStorage-bkucn6ign34i.0]: UPDATE_FAILED  resources[0]:
RetryError: resources.CephStorage: RetryError[<Future at 0xdd62550
state=finished returned bool>]
2017-04-12 00:26:34.436989 | 2017-04-12 00:26:29Z
[overcloud-CephStorage-bkucn6ign34i]: UPDATE_FAILED  resources[0]:
RetryError: resources.CephStorage: RetryError[<Future at 0xdd62550
state=finished returned bool>]
2017-04-12 00:26:34.437078 | 2017-04-12 00:26:30Z
[overcloud-Controller-3lf3jauv4cbc-0-ydowkb3nwsso.Controller]:
UPDATE_FAILED  RetryError: resources.Controller: RetryError[<Future at
0xdc79b50 state=finished returned bool>]
2017-04-12 00:26:34.437173 | 2017-04-12 00:26:30Z
[overcloud-Controller-3lf3jauv4cbc-0-ydowkb3nwsso]: UPDATE_FAILED
RetryError: resources.Controller: RetryError[<Future at 0xdc79b50
state=finished returned bool>]
2017-04-12 00:26:34.437269 | 2017-04-12 00:26:30Z [CephStorage]:
UPDATE_FAILED  resources.CephStorage: resources[0]: RetryError:
resources.CephStorage: RetryError[<Future at 0xdd62550 state=finished
returned bool>]

--
Dan Sneddon         |  Senior Principal Software Engineer
[hidden email]redhat.com/openstack
dsneddon:irc        |  @dxs:twitter

__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



--
Regards,
Rabi Misra


__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: [hidden email]?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [TripleO][Heat] Conditionally passing properties in Heat

Dan Sneddon
On 04/13/2017 12:01 AM, Rabi Mishra wrote:

> On Thu, Apr 13, 2017 at 2:14 AM, Dan Sneddon <[hidden email]
> <mailto:[hidden email]>> wrote:
>
>     On 04/12/2017 01:22 PM, Thomas Herve wrote:
>     > On Wed, Apr 12, 2017 at 9:00 PM, Dan Sneddon <[hidden email]
>     <mailto:[hidden email]>> wrote:
>     >> I'm implementing predictable control plane IPs for spine/leaf,
>     and I'm
>     >> running into a problem implementing this in the TripleO Heat
>     templates.
>     >>
>     >> I have a review in progress [1] that works, but fails on upgrade,
>     so I'm
>     >> looking for an alternative approach. I'm trying to influence the IP
>     >> address that is selected for overcloud nodes' Control Plane IP.
>     Here is
>     >> the current construct:
>     >>
>     >>   Controller:
>     >>     type: OS::TripleO::Server
>     >>     metadata:
>     >>       os-collect-config:
>     >>         command: {get_param: ConfigCommand}
>     >>     properties:
>     >>       image: {get_param: controllerImage}
>     >>       image_update_policy: {get_param: ImageUpdatePolicy}
>     >>       flavor: {get_param: OvercloudControlFlavor}
>     >>       key_name: {get_param: KeyName}
>     >>       networks:
>     >>         - network: ctlplane  # <- Here's where the port is created
>     >>
>     >> If I add fixed_ip: to the networks element at the end of the above, I
>     >> can select an IP address from the 'ctlplane' network, like this:
>     >>
>     >>       networks:
>     >>         - network: ctlplane
>     >>           fixed_ip: {get_attr: [ControlPlanePort, ip_address]}
>     >>
>     >> But the problem is that if I pass a blank string to fixed_ip, I
>     get an
>     >> error on deployment. This means that the old behavior of
>     automatically
>     >> selecting an IP doesn't work.
>     >>
>     >> I thought I has solved this by passing an external Neutron port,
>     like this:
>     >>
>     >>       networks:
>     >>         - network: ctlplane
>     >>           port: {get_attr: [ControlPlanePort, port_id]}
>     >>
>     >> Which works for deployments, but that fails on upgrades, since the
>     >> original port was created as part of the Nova::Server resource,
>     instead
>     >> of being an external resource.
>     >
>     > Can you detail how it fails? I was under the impression we never
>     > replaced servers no matter what (or we try to do that, at least). Is
>     > the issue that your new port is not the correct one?
>     >
>     >> I'm now looking for a way to use Heat conditionals to apply the
>     fixed_ip
>     >> only if the value is not unset. Looking at the intrinsic
>     functions [2],
>     >> I don't see a way to do this. Is what I'm trying to do with Heat
>     possible?
>     >
>     > You should be able to write something like that (not tested):
>     >
>     > networks:
>     >   if:
>     >     - <my condition>
>     >     - network: ctlplane
>     >       fixed_ip: {get_attr: [ControlPlanePort, ip_address]}
>     >     - network: ctlplane
>     >
>     > The question is how to define your condition. Maybe:
>     >
>     > conditions:
>     >   fixed_ip_condition:
>     >      not:
>     >         equals:
>     >           - {get_attr: [ControlPlanePort, ip_address]}
>     >           - ''
>     >
>     > To get back to the problem you stated first.
>     >
>     >
>     >> Another option I'm exploring is conditionally applying resources. It
>     >> appears that would require duplicating the entire TripleO::Server
>     stanza
>     >> in *-role.yaml so that there is one that uses fixed_ip and one
>     that does
>     >> not. Which one is applied would be based on a condition that tested
>     >> whether fixed_ip was blank or not. The downside of that is that
>     it would
>     >> make the role definition confusing because there would be a large
>     >> resource that was implemented twice, with only one line difference
>     >> between them.
>     >
>     > You can define properties with conditions, so you shouldn't need to
>     > rewrite everything.
>     >
>
>     Thomas,
>
>     Thanks, I will try your suggestions and that should get me closer.
>
>     The full error log is available here:
>     http://logs.openstack.org/78/413278/11/check-tripleo/gate-tripleo-ci-centos-7-ovb-updates/8d91762/console.html
>     <http://logs.openstack.org/78/413278/11/check-tripleo/gate-tripleo-ci-centos-7-ovb-updates/8d91762/console.html>
>
> We do an interface_detach/attach when a port is replaced.
> It seems to be failing[1] as this is not implemented for
> ironic/baremetal driver.  I could see a patch[2] to add that
> functionality though.
>
> [1]
> http://logs.openstack.org/78/413278/11/check-tripleo/gate-tripleo-ci-centos-7-ovb-updates/8d91762/logs/undercloud/var/log/nova/nova-compute.txt.gz#_2017-04-12_00_26_15_475
>
> [2] https://review.openstack.org/#/c/419975/
>
> We retry a few times to check whether the detach/attach is complete(it's
> an async operation in nova and takes time), so the cryptic error below
> is coming from tenacity library which fails after the configured number
> of attempts.
>
>     Here are the errors I am getting:
>
>     2017-04-12 00:26:34.436655 | 2017-04-12 00:26:29Z
>     [overcloud-CephStorage-bkucn6ign34i-0-2yq2jbtwuu7k.CephStorage]:
>     UPDATE_FAILED  RetryError: resources.CephStorage: RetryError[<Future at
>     0xdd62550 state=finished returned bool>]
>     2017-04-12 00:26:34.436808 | 2017-04-12 00:26:29Z
>     [overcloud-CephStorage-bkucn6ign34i-0-2yq2jbtwuu7k]: UPDATE_FAILED
>     RetryError: resources.CephStorage: RetryError[<Future at 0xdd62550
>     state=finished returned bool>]
>     2017-04-12 00:26:34.436903 | 2017-04-12 00:26:29Z
>     [overcloud-CephStorage-bkucn6ign34i.0]: UPDATE_FAILED  resources[0]:
>     RetryError: resources.CephStorage: RetryError[<Future at 0xdd62550
>     state=finished returned bool>]
>     2017-04-12 00:26:34.436989 | 2017-04-12 00:26:29Z
>     [overcloud-CephStorage-bkucn6ign34i]: UPDATE_FAILED  resources[0]:
>     RetryError: resources.CephStorage: RetryError[<Future at 0xdd62550
>     state=finished returned bool>]
>     2017-04-12 00:26:34.437078 | 2017-04-12 00:26:30Z
>     [overcloud-Controller-3lf3jauv4cbc-0-ydowkb3nwsso.Controller]:
>     UPDATE_FAILED  RetryError: resources.Controller: RetryError[<Future at
>     0xdc79b50 state=finished returned bool>]
>     2017-04-12 00:26:34.437173 | 2017-04-12 00:26:30Z
>     [overcloud-Controller-3lf3jauv4cbc-0-ydowkb3nwsso]: UPDATE_FAILED
>     RetryError: resources.Controller: RetryError[<Future at 0xdc79b50
>     state=finished returned bool>]
>     2017-04-12 00:26:34.437269 | 2017-04-12 00:26:30Z [CephStorage]:
>     UPDATE_FAILED  resources.CephStorage: resources[0]: RetryError:
>     resources.CephStorage: RetryError[<Future at 0xdd62550 state=finished
>     returned bool>]
>
>     --
>     Dan Sneddon         |  Senior Principal Software Engineer
>     [hidden email] <mailto:[hidden email]> |
>     redhat.com/openstack <http://redhat.com/openstack>
>     dsneddon:irc        |  @dxs:twitter
>
> --
> Regards,
> Rabi Misra

Rabi,

Thanks for the explanation on why the port replacement isn't working.

Unfortunately, I tried the recommendation that Thomas Herve made about
using conditionals, but that failed. It appears that you can't use a
get_attr inside of a conditional statement, I get an error to that
effect. I can use a get_param, but that doesn't help me check a value in
a nested stack.

So in order for me to achieve the goal of being able to conditionally
assign the IP automatically or from a fixed_ip, I need either a Heat
feature (to resolve get_attr inside of conditional statements), or the
Nova patch to implement attach/detach in Ironic.

Thanks to both of you for the assistance.

--
Dan Sneddon         |  Senior Principal Software Engineer
[hidden email] |  redhat.com/openstack
dsneddon:irc        |  @dxs:twitter

__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: [hidden email]?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [TripleO][Heat] Conditionally passing properties in Heat

Steven Hardy
On Wed, Apr 19, 2017 at 02:51:28PM -0700, Dan Sneddon wrote:

> On 04/13/2017 12:01 AM, Rabi Mishra wrote:
> > On Thu, Apr 13, 2017 at 2:14 AM, Dan Sneddon <[hidden email]
> > <mailto:[hidden email]>> wrote:
> >
> >     On 04/12/2017 01:22 PM, Thomas Herve wrote:
> >     > On Wed, Apr 12, 2017 at 9:00 PM, Dan Sneddon <[hidden email]
> >     <mailto:[hidden email]>> wrote:
> >     >> I'm implementing predictable control plane IPs for spine/leaf,
> >     and I'm
> >     >> running into a problem implementing this in the TripleO Heat
> >     templates.
> >     >>
> >     >> I have a review in progress [1] that works, but fails on upgrade,
> >     so I'm
> >     >> looking for an alternative approach. I'm trying to influence the IP
> >     >> address that is selected for overcloud nodes' Control Plane IP.
> >     Here is
> >     >> the current construct:
> >     >>
> >     >>   Controller:
> >     >>     type: OS::TripleO::Server
> >     >>     metadata:
> >     >>       os-collect-config:
> >     >>         command: {get_param: ConfigCommand}
> >     >>     properties:
> >     >>       image: {get_param: controllerImage}
> >     >>       image_update_policy: {get_param: ImageUpdatePolicy}
> >     >>       flavor: {get_param: OvercloudControlFlavor}
> >     >>       key_name: {get_param: KeyName}
> >     >>       networks:
> >     >>         - network: ctlplane  # <- Here's where the port is created
> >     >>
> >     >> If I add fixed_ip: to the networks element at the end of the above, I
> >     >> can select an IP address from the 'ctlplane' network, like this:
> >     >>
> >     >>       networks:
> >     >>         - network: ctlplane
> >     >>           fixed_ip: {get_attr: [ControlPlanePort, ip_address]}
> >     >>
> >     >> But the problem is that if I pass a blank string to fixed_ip, I
> >     get an
> >     >> error on deployment. This means that the old behavior of
> >     automatically
> >     >> selecting an IP doesn't work.
> >     >>
> >     >> I thought I has solved this by passing an external Neutron port,
> >     like this:
> >     >>
> >     >>       networks:
> >     >>         - network: ctlplane
> >     >>           port: {get_attr: [ControlPlanePort, port_id]}
> >     >>
> >     >> Which works for deployments, but that fails on upgrades, since the
> >     >> original port was created as part of the Nova::Server resource,
> >     instead
> >     >> of being an external resource.
> >     >
> >     > Can you detail how it fails? I was under the impression we never
> >     > replaced servers no matter what (or we try to do that, at least). Is
> >     > the issue that your new port is not the correct one?
> >     >
> >     >> I'm now looking for a way to use Heat conditionals to apply the
> >     fixed_ip
> >     >> only if the value is not unset. Looking at the intrinsic
> >     functions [2],
> >     >> I don't see a way to do this. Is what I'm trying to do with Heat
> >     possible?
> >     >
> >     > You should be able to write something like that (not tested):
> >     >
> >     > networks:
> >     >   if:
> >     >     - <my condition>
> >     >     - network: ctlplane
> >     >       fixed_ip: {get_attr: [ControlPlanePort, ip_address]}
> >     >     - network: ctlplane
> >     >
> >     > The question is how to define your condition. Maybe:
> >     >
> >     > conditions:
> >     >   fixed_ip_condition:
> >     >      not:
> >     >         equals:
> >     >           - {get_attr: [ControlPlanePort, ip_address]}
> >     >           - ''
> >     >
> >     > To get back to the problem you stated first.
> >     >
> >     >
> >     >> Another option I'm exploring is conditionally applying resources. It
> >     >> appears that would require duplicating the entire TripleO::Server
> >     stanza
> >     >> in *-role.yaml so that there is one that uses fixed_ip and one
> >     that does
> >     >> not. Which one is applied would be based on a condition that tested
> >     >> whether fixed_ip was blank or not. The downside of that is that
> >     it would
> >     >> make the role definition confusing because there would be a large
> >     >> resource that was implemented twice, with only one line difference
> >     >> between them.
> >     >
> >     > You can define properties with conditions, so you shouldn't need to
> >     > rewrite everything.
> >     >
> >
> >     Thomas,
> >
> >     Thanks, I will try your suggestions and that should get me closer.
> >
> >     The full error log is available here:
> >     http://logs.openstack.org/78/413278/11/check-tripleo/gate-tripleo-ci-centos-7-ovb-updates/8d91762/console.html
> >     <http://logs.openstack.org/78/413278/11/check-tripleo/gate-tripleo-ci-centos-7-ovb-updates/8d91762/console.html>
> >
> > We do an interface_detach/attach when a port is replaced.
> > It seems to be failing[1] as this is not implemented for
> > ironic/baremetal driver.  I could see a patch[2] to add that
> > functionality though.
> >
> > [1]
> > http://logs.openstack.org/78/413278/11/check-tripleo/gate-tripleo-ci-centos-7-ovb-updates/8d91762/logs/undercloud/var/log/nova/nova-compute.txt.gz#_2017-04-12_00_26_15_475
> >
> > [2] https://review.openstack.org/#/c/419975/
> >
> > We retry a few times to check whether the detach/attach is complete(it's
> > an async operation in nova and takes time), so the cryptic error below
> > is coming from tenacity library which fails after the configured number
> > of attempts.
> >
> >     Here are the errors I am getting:
> >
> >     2017-04-12 00:26:34.436655 | 2017-04-12 00:26:29Z
> >     [overcloud-CephStorage-bkucn6ign34i-0-2yq2jbtwuu7k.CephStorage]:
> >     UPDATE_FAILED  RetryError: resources.CephStorage: RetryError[<Future at
> >     0xdd62550 state=finished returned bool>]
> >     2017-04-12 00:26:34.436808 | 2017-04-12 00:26:29Z
> >     [overcloud-CephStorage-bkucn6ign34i-0-2yq2jbtwuu7k]: UPDATE_FAILED
> >     RetryError: resources.CephStorage: RetryError[<Future at 0xdd62550
> >     state=finished returned bool>]
> >     2017-04-12 00:26:34.436903 | 2017-04-12 00:26:29Z
> >     [overcloud-CephStorage-bkucn6ign34i.0]: UPDATE_FAILED  resources[0]:
> >     RetryError: resources.CephStorage: RetryError[<Future at 0xdd62550
> >     state=finished returned bool>]
> >     2017-04-12 00:26:34.436989 | 2017-04-12 00:26:29Z
> >     [overcloud-CephStorage-bkucn6ign34i]: UPDATE_FAILED  resources[0]:
> >     RetryError: resources.CephStorage: RetryError[<Future at 0xdd62550
> >     state=finished returned bool>]
> >     2017-04-12 00:26:34.437078 | 2017-04-12 00:26:30Z
> >     [overcloud-Controller-3lf3jauv4cbc-0-ydowkb3nwsso.Controller]:
> >     UPDATE_FAILED  RetryError: resources.Controller: RetryError[<Future at
> >     0xdc79b50 state=finished returned bool>]
> >     2017-04-12 00:26:34.437173 | 2017-04-12 00:26:30Z
> >     [overcloud-Controller-3lf3jauv4cbc-0-ydowkb3nwsso]: UPDATE_FAILED
> >     RetryError: resources.Controller: RetryError[<Future at 0xdc79b50
> >     state=finished returned bool>]
> >     2017-04-12 00:26:34.437269 | 2017-04-12 00:26:30Z [CephStorage]:
> >     UPDATE_FAILED  resources.CephStorage: resources[0]: RetryError:
> >     resources.CephStorage: RetryError[<Future at 0xdd62550 state=finished
> >     returned bool>]
> >
> >     --
> >     Dan Sneddon         |  Senior Principal Software Engineer
> >     [hidden email] <mailto:[hidden email]> |
> >     redhat.com/openstack <http://redhat.com/openstack>
> >     dsneddon:irc        |  @dxs:twitter
> >
> > --
> > Regards,
> > Rabi Misra
>
> Rabi,
>
> Thanks for the explanation on why the port replacement isn't working.
>
> Unfortunately, I tried the recommendation that Thomas Herve made about
> using conditionals, but that failed. It appears that you can't use a
> get_attr inside of a conditional statement, I get an error to that
> effect. I can use a get_param, but that doesn't help me check a value in
> a nested stack.

Yes I noticed the same thing recently, which is a limitation of conditions
that IMO we should look at fixing - everywhere else in HOT templates
get_param and get_attr can be used interchangably.

As a workaround could you move the conditional into the port resource
nested stack?  E.g if you returned the list we pass to "networks" for the
server resource from the new ControlPlanePort nested stack perhaps, and put
whatever logic we need to generate it inside the port nested stack?

Attributes from e.g *-role.yaml would then be parameters in the nested
stack, so you could do the conditional as Thomas suggested, or perhaps
it's even simpler if you can just return different data from the
ctlplane.yaml and ctlplane_from_pool.yaml templates?

Steve

__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: [hidden email]?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [TripleO][Heat] Conditionally passing properties in Heat

Zane Bitter
On 20/04/17 03:37, Steven Hardy wrote:

> On Wed, Apr 19, 2017 at 02:51:28PM -0700, Dan Sneddon wrote:
>> Unfortunately, I tried the recommendation that Thomas Herve made about
>> using conditionals, but that failed. It appears that you can't use a
>> get_attr inside of a conditional statement, I get an error to that
>> effect. I can use a get_param, but that doesn't help me check a value in
>> a nested stack.
>
> Yes I noticed the same thing recently, which is a limitation of conditions
> that IMO we should look at fixing - everywhere else in HOT templates
> get_param and get_attr can be used interchangably.

Right, but conditions are essentially preprocessor macros. They can't
depend on data that is not available until runtime.

- ZB

__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: [hidden email]?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [TripleO][Heat] Conditionally passing properties in Heat

Dan Sneddon
In reply to this post by Steven Hardy
On 04/20/2017 12:37 AM, Steven Hardy wrote:

> On Wed, Apr 19, 2017 at 02:51:28PM -0700, Dan Sneddon wrote:
>> On 04/13/2017 12:01 AM, Rabi Mishra wrote:
>>> On Thu, Apr 13, 2017 at 2:14 AM, Dan Sneddon <[hidden email]
>>> <mailto:[hidden email]>> wrote:
>>>
>>>     On 04/12/2017 01:22 PM, Thomas Herve wrote:
>>>     > On Wed, Apr 12, 2017 at 9:00 PM, Dan Sneddon <[hidden email]
>>>     <mailto:[hidden email]>> wrote:
>>>     >> I'm implementing predictable control plane IPs for spine/leaf,
>>>     and I'm
>>>     >> running into a problem implementing this in the TripleO Heat
>>>     templates.
>>>     >>
>>>     >> I have a review in progress [1] that works, but fails on upgrade,
>>>     so I'm
>>>     >> looking for an alternative approach. I'm trying to influence the IP
>>>     >> address that is selected for overcloud nodes' Control Plane IP.
>>>     Here is
>>>     >> the current construct:
>>>     >>
>>>     >>   Controller:
>>>     >>     type: OS::TripleO::Server
>>>     >>     metadata:
>>>     >>       os-collect-config:
>>>     >>         command: {get_param: ConfigCommand}
>>>     >>     properties:
>>>     >>       image: {get_param: controllerImage}
>>>     >>       image_update_policy: {get_param: ImageUpdatePolicy}
>>>     >>       flavor: {get_param: OvercloudControlFlavor}
>>>     >>       key_name: {get_param: KeyName}
>>>     >>       networks:
>>>     >>         - network: ctlplane  # <- Here's where the port is created
>>>     >>
>>>     >> If I add fixed_ip: to the networks element at the end of the above, I
>>>     >> can select an IP address from the 'ctlplane' network, like this:
>>>     >>
>>>     >>       networks:
>>>     >>         - network: ctlplane
>>>     >>           fixed_ip: {get_attr: [ControlPlanePort, ip_address]}
>>>     >>
>>>     >> But the problem is that if I pass a blank string to fixed_ip, I
>>>     get an
>>>     >> error on deployment. This means that the old behavior of
>>>     automatically
>>>     >> selecting an IP doesn't work.
>>>     >>
>>>     >> I thought I has solved this by passing an external Neutron port,
>>>     like this:
>>>     >>
>>>     >>       networks:
>>>     >>         - network: ctlplane
>>>     >>           port: {get_attr: [ControlPlanePort, port_id]}
>>>     >>
>>>     >> Which works for deployments, but that fails on upgrades, since the
>>>     >> original port was created as part of the Nova::Server resource,
>>>     instead
>>>     >> of being an external resource.
>>>     >
>>>     > Can you detail how it fails? I was under the impression we never
>>>     > replaced servers no matter what (or we try to do that, at least). Is
>>>     > the issue that your new port is not the correct one?
>>>     >
>>>     >> I'm now looking for a way to use Heat conditionals to apply the
>>>     fixed_ip
>>>     >> only if the value is not unset. Looking at the intrinsic
>>>     functions [2],
>>>     >> I don't see a way to do this. Is what I'm trying to do with Heat
>>>     possible?
>>>     >
>>>     > You should be able to write something like that (not tested):
>>>     >
>>>     > networks:
>>>     >   if:
>>>     >     - <my condition>
>>>     >     - network: ctlplane
>>>     >       fixed_ip: {get_attr: [ControlPlanePort, ip_address]}
>>>     >     - network: ctlplane
>>>     >
>>>     > The question is how to define your condition. Maybe:
>>>     >
>>>     > conditions:
>>>     >   fixed_ip_condition:
>>>     >      not:
>>>     >         equals:
>>>     >           - {get_attr: [ControlPlanePort, ip_address]}
>>>     >           - ''
>>>     >
>>>     > To get back to the problem you stated first.
>>>     >
>>>     >
>>>     >> Another option I'm exploring is conditionally applying resources. It
>>>     >> appears that would require duplicating the entire TripleO::Server
>>>     stanza
>>>     >> in *-role.yaml so that there is one that uses fixed_ip and one
>>>     that does
>>>     >> not. Which one is applied would be based on a condition that tested
>>>     >> whether fixed_ip was blank or not. The downside of that is that
>>>     it would
>>>     >> make the role definition confusing because there would be a large
>>>     >> resource that was implemented twice, with only one line difference
>>>     >> between them.
>>>     >
>>>     > You can define properties with conditions, so you shouldn't need to
>>>     > rewrite everything.
>>>     >
>>>
>>>     Thomas,
>>>
>>>     Thanks, I will try your suggestions and that should get me closer.
>>>
>>>     The full error log is available here:
>>>     http://logs.openstack.org/78/413278/11/check-tripleo/gate-tripleo-ci-centos-7-ovb-updates/8d91762/console.html
>>>     <http://logs.openstack.org/78/413278/11/check-tripleo/gate-tripleo-ci-centos-7-ovb-updates/8d91762/console.html>
>>>
>>> We do an interface_detach/attach when a port is replaced.
>>> It seems to be failing[1] as this is not implemented for
>>> ironic/baremetal driver.  I could see a patch[2] to add that
>>> functionality though.
>>>
>>> [1]
>>> http://logs.openstack.org/78/413278/11/check-tripleo/gate-tripleo-ci-centos-7-ovb-updates/8d91762/logs/undercloud/var/log/nova/nova-compute.txt.gz#_2017-04-12_00_26_15_475
>>>
>>> [2] https://review.openstack.org/#/c/419975/
>>>
>>> We retry a few times to check whether the detach/attach is complete(it's
>>> an async operation in nova and takes time), so the cryptic error below
>>> is coming from tenacity library which fails after the configured number
>>> of attempts.
>>>
>>>     Here are the errors I am getting:
>>>
>>>     2017-04-12 00:26:34.436655 | 2017-04-12 00:26:29Z
>>>     [overcloud-CephStorage-bkucn6ign34i-0-2yq2jbtwuu7k.CephStorage]:
>>>     UPDATE_FAILED  RetryError: resources.CephStorage: RetryError[<Future at
>>>     0xdd62550 state=finished returned bool>]
>>>     2017-04-12 00:26:34.436808 | 2017-04-12 00:26:29Z
>>>     [overcloud-CephStorage-bkucn6ign34i-0-2yq2jbtwuu7k]: UPDATE_FAILED
>>>     RetryError: resources.CephStorage: RetryError[<Future at 0xdd62550
>>>     state=finished returned bool>]
>>>     2017-04-12 00:26:34.436903 | 2017-04-12 00:26:29Z
>>>     [overcloud-CephStorage-bkucn6ign34i.0]: UPDATE_FAILED  resources[0]:
>>>     RetryError: resources.CephStorage: RetryError[<Future at 0xdd62550
>>>     state=finished returned bool>]
>>>     2017-04-12 00:26:34.436989 | 2017-04-12 00:26:29Z
>>>     [overcloud-CephStorage-bkucn6ign34i]: UPDATE_FAILED  resources[0]:
>>>     RetryError: resources.CephStorage: RetryError[<Future at 0xdd62550
>>>     state=finished returned bool>]
>>>     2017-04-12 00:26:34.437078 | 2017-04-12 00:26:30Z
>>>     [overcloud-Controller-3lf3jauv4cbc-0-ydowkb3nwsso.Controller]:
>>>     UPDATE_FAILED  RetryError: resources.Controller: RetryError[<Future at
>>>     0xdc79b50 state=finished returned bool>]
>>>     2017-04-12 00:26:34.437173 | 2017-04-12 00:26:30Z
>>>     [overcloud-Controller-3lf3jauv4cbc-0-ydowkb3nwsso]: UPDATE_FAILED
>>>     RetryError: resources.Controller: RetryError[<Future at 0xdc79b50
>>>     state=finished returned bool>]
>>>     2017-04-12 00:26:34.437269 | 2017-04-12 00:26:30Z [CephStorage]:
>>>     UPDATE_FAILED  resources.CephStorage: resources[0]: RetryError:
>>>     resources.CephStorage: RetryError[<Future at 0xdd62550 state=finished
>>>     returned bool>]
>>>
>>>     --
>>>     Dan Sneddon         |  Senior Principal Software Engineer
>>>     [hidden email] <mailto:[hidden email]> |
>>>     redhat.com/openstack <http://redhat.com/openstack>
>>>     dsneddon:irc        |  @dxs:twitter
>>>
>>> --
>>> Regards,
>>> Rabi Misra
>>
>> Rabi,
>>
>> Thanks for the explanation on why the port replacement isn't working.
>>
>> Unfortunately, I tried the recommendation that Thomas Herve made about
>> using conditionals, but that failed. It appears that you can't use a
>> get_attr inside of a conditional statement, I get an error to that
>> effect. I can use a get_param, but that doesn't help me check a value in
>> a nested stack.
>
> Yes I noticed the same thing recently, which is a limitation of conditions
> that IMO we should look at fixing - everywhere else in HOT templates
> get_param and get_attr can be used interchangably.
>
> As a workaround could you move the conditional into the port resource
> nested stack?  E.g if you returned the list we pass to "networks" for the
> server resource from the new ControlPlanePort nested stack perhaps, and put
> whatever logic we need to generate it inside the port nested stack?
>
> Attributes from e.g *-role.yaml would then be parameters in the nested
> stack, so you could do the conditional as Thomas suggested, or perhaps
> it's even simpler if you can just return different data from the
> ctlplane.yaml and ctlplane_from_pool.yaml templates?
>
> Steve

Steve,

I tried moving the generation of the network parameters for the role
into the port. That way, I shouldn't need a conditional because I can
select which template to use (with or without a fixed_ip). That didn't
work either, it doesn't appear that I can pass a list as a parameter to
the "networks" property of TripleO::Server.

"controller-role.yaml":
  Controller:
    type: OS::TripleO::Server
    metadata:
      os-collect-config:
        command: {get_param: ConfigCommand}
    properties:
      image: {get_param: controllerImage}
      image_update_policy: {get_param: ImageUpdatePolicy}
      flavor: {get_param: OvercloudControlFlavor}
      key_name: {get_param: KeyName}
      networks: {get_attr: [ControlPlanePort, port_info]}

Then, the ControlPlanePort has this:
"ctlplane.yaml":
outputs:
  port_info:
    description: The newtork name to use for port
    value:
      - network: {get_param: ControlPlaneNetName}

OR "ctlplane_from_pool.yaml" defines the same value as:

  port_info:
    description: The newtork name and fixed_ip to use for port
    value:
      - network: {get_param: ControlPlaneNetName}
        fixed_ip: {get_param: [IPPool, {get_param: ControlPlaneNetName},
{get_param: NodeIndex}]}

Unfortunately, this fails. For example, when passing only the network:

"""
GET call to orchestration for
http://172.20.0.1:8004/v1/f6d7244fe69c43359c293d2c9edf0ce5/stacks/over
cloud-ObjectStorageIpListMap-kwmthuphny55/1d8b2ef4-8223-412b-b32f-6e00a9651478/resources
used reques
t id req-50c24b3f-2118-444b-878f-94e7d0a3b949
overcloud.Controller.0.Controller:
  resource_type: OS::TripleO::Server
  physical_resource_id:
  status: CREATE_FAILED
  status_reason: |
    BadRequest: resources.Controller: Bad networks format: network uuid
is not in proper format (ctlplane) (HTTP 400) (Request-ID:
req-e3da2bf7-91bb-4902-bdcc-0ab11d1e2e09)
"""

Is there a way that I can pass a list between a nested resource (port)
and the parent stack (the role in this case), so I can pass either a
network or a network plus fixed IP? I'm not sure why this is failing,
since simply passing ctlplane works in the original role definition:

  Controller:
    type: OS::TripleO::Server
    metadata:
      os-collect-config:
        command: {get_param: ConfigCommand}
    properties:
      image: {get_param: controllerImage}
      image_update_policy: {get_param: ImageUpdatePolicy}
      flavor: {get_param: OvercloudControlFlavor}
      key_name: {get_param: KeyName}
      networks:
        - network: ctlplane

--
Dan Sneddon         |  Senior Principal Software Engineer
[hidden email] |  redhat.com/openstack
dsneddon:irc        |  @dxs:twitter

__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: [hidden email]?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [TripleO][Heat] Conditionally passing properties in Heat

Harald Jensås
On Thu, 2017-04-20 at 16:11 -0700, Dan Sneddon wrote:

> On 04/20/2017 12:37 AM, Steven Hardy wrote:
> > On Wed, Apr 19, 2017 at 02:51:28PM -0700, Dan Sneddon wrote:
> > > On 04/13/2017 12:01 AM, Rabi Mishra wrote:
> > > > On Thu, Apr 13, 2017 at 2:14 AM, Dan Sneddon <[hidden email]
> > > > om
> > > > <mailto:[hidden email]>> wrote:
> > > >
> > > >     On 04/12/2017 01:22 PM, Thomas Herve wrote:
> > > >     > On Wed, Apr 12, 2017 at 9:00 PM, Dan Sneddon <dsneddon@re
> > > > dhat.com
> > > >     <mailto:[hidden email]>> wrote:
> > > >     >> I'm implementing predictable control plane IPs for
> > > > spine/leaf,
> > > >     and I'm
> > > >     >> running into a problem implementing this in the TripleO
> > > > Heat
> > > >     templates.
> > > >     >>
> > > >     >> I have a review in progress [1] that works, but fails on
> > > > upgrade,
> > > >     so I'm
> > > >     >> looking for an alternative approach. I'm trying to
> > > > influence the IP
> > > >     >> address that is selected for overcloud nodes' Control
> > > > Plane IP.
> > > >     Here is
> > > >     >> the current construct:
> > > >     >>
> > > >     >>   Controller:
> > > >     >>     type: OS::TripleO::Server
> > > >     >>     metadata:
> > > >     >>       os-collect-config:
> > > >     >>         command: {get_param: ConfigCommand}
> > > >     >>     properties:
> > > >     >>       image: {get_param: controllerImage}
> > > >     >>       image_update_policy: {get_param:
> > > > ImageUpdatePolicy}
> > > >     >>       flavor: {get_param: OvercloudControlFlavor}
> > > >     >>       key_name: {get_param: KeyName}
> > > >     >>       networks:
> > > >     >>         - network: ctlplane  # <- Here's where the port
> > > > is created
> > > >     >>
> > > >     >> If I add fixed_ip: to the networks element at the end of
> > > > the above, I
> > > >     >> can select an IP address from the 'ctlplane' network,
> > > > like this:
> > > >     >>
> > > >     >>       networks:
> > > >     >>         - network: ctlplane
> > > >     >>           fixed_ip: {get_attr: [ControlPlanePort,
> > > > ip_address]}
> > > >     >>
> > > >     >> But the problem is that if I pass a blank string to
> > > > fixed_ip, I
> > > >     get an
> > > >     >> error on deployment. This means that the old behavior of
> > > >     automatically
> > > >     >> selecting an IP doesn't work.
> > > >     >>
> > > >     >> I thought I has solved this by passing an external
> > > > Neutron port,
> > > >     like this:
> > > >     >>
> > > >     >>       networks:
> > > >     >>         - network: ctlplane
> > > >     >>           port: {get_attr: [ControlPlanePort, port_id]}
> > > >     >>
> > > >     >> Which works for deployments, but that fails on upgrades,
> > > > since the
> > > >     >> original port was created as part of the Nova::Server
> > > > resource,
> > > >     instead
> > > >     >> of being an external resource.
> > > >     >
> > > >     > Can you detail how it fails? I was under the impression
> > > > we never
> > > >     > replaced servers no matter what (or we try to do that, at
> > > > least). Is
> > > >     > the issue that your new port is not the correct one?
> > > >     >
> > > >     >> I'm now looking for a way to use Heat conditionals to
> > > > apply the
> > > >     fixed_ip
> > > >     >> only if the value is not unset. Looking at the intrinsic
> > > >     functions [2],
> > > >     >> I don't see a way to do this. Is what I'm trying to do
> > > > with Heat
> > > >     possible?
> > > >     >
> > > >     > You should be able to write something like that (not
> > > > tested):
> > > >     >
> > > >     > networks:
> > > >     >   if:
> > > >     >     - <my condition>
> > > >     >     - network: ctlplane
> > > >     >       fixed_ip: {get_attr: [ControlPlanePort,
> > > > ip_address]}
> > > >     >     - network: ctlplane
> > > >     >
> > > >     > The question is how to define your condition. Maybe:
> > > >     >
> > > >     > conditions:
> > > >     >   fixed_ip_condition:
> > > >     >      not:
> > > >     >         equals:
> > > >     >           - {get_attr: [ControlPlanePort, ip_address]}
> > > >     >           - ''
> > > >     >
> > > >     > To get back to the problem you stated first.
> > > >     >
> > > >     >
> > > >     >> Another option I'm exploring is conditionally applying
> > > > resources. It
> > > >     >> appears that would require duplicating the entire
> > > > TripleO::Server
> > > >     stanza
> > > >     >> in *-role.yaml so that there is one that uses fixed_ip
> > > > and one
> > > >     that does
> > > >     >> not. Which one is applied would be based on a condition
> > > > that tested
> > > >     >> whether fixed_ip was blank or not. The downside of that
> > > > is that
> > > >     it would
> > > >     >> make the role definition confusing because there would
> > > > be a large
> > > >     >> resource that was implemented twice, with only one line
> > > > difference
> > > >     >> between them.
> > > >     >
> > > >     > You can define properties with conditions, so you
> > > > shouldn't need to
> > > >     > rewrite everything.
> > > >     >
> > > >
> > > >     Thomas,
> > > >
> > > >     Thanks, I will try your suggestions and that should get me
> > > > closer.
> > > >
> > > >     The full error log is available here:
> > > >     http://logs.openstack.org/78/413278/11/check-tripleo/gate-t
> > > > ripleo-ci-centos-7-ovb-updates/8d91762/console.html
> > > >     <http://logs.openstack.org/78/413278/11/check-tripleo/gate-
> > > > tripleo-ci-centos-7-ovb-updates/8d91762/console.html>
> > > >
> > > > We do an interface_detach/attach when a port is replaced.
> > > > It seems to be failing[1] as this is not implemented for
> > > > ironic/baremetal driver.  I could see a patch[2] to add that
> > > > functionality though.
> > > >
> > > > [1]
> > > > http://logs.openstack.org/78/413278/11/check-tripleo/gate-tripl
> > > > eo-ci-centos-7-ovb-
> > > > updates/8d91762/logs/undercloud/var/log/nova/nova-
> > > > compute.txt.gz#_2017-04-12_00_26_15_475
> > > >
> > > > [2] https://review.openstack.org/#/c/419975/
> > > >
> > > > We retry a few times to check whether the detach/attach is
> > > > complete(it's
> > > > an async operation in nova and takes time), so the cryptic
> > > > error below
> > > > is coming from tenacity library which fails after the
> > > > configured number
> > > > of attempts.
> > > >
> > > >     Here are the errors I am getting:
> > > >
> > > >     2017-04-12 00:26:34.436655 | 2017-04-12 00:26:29Z
> > > >     [overcloud-CephStorage-bkucn6ign34i-0-
> > > > 2yq2jbtwuu7k.CephStorage]:
> > > >     UPDATE_FAILED  RetryError: resources.CephStorage:
> > > > RetryError[<Future at
> > > >     0xdd62550 state=finished returned bool>]
> > > >     2017-04-12 00:26:34.436808 | 2017-04-12 00:26:29Z
> > > >     [overcloud-CephStorage-bkucn6ign34i-0-2yq2jbtwuu7k]:
> > > > UPDATE_FAILED
> > > >     RetryError: resources.CephStorage: RetryError[<Future at
> > > > 0xdd62550
> > > >     state=finished returned bool>]
> > > >     2017-04-12 00:26:34.436903 | 2017-04-12 00:26:29Z
> > > >     [overcloud-CephStorage-bkucn6ign34i.0]:
> > > > UPDATE_FAILED  resources[0]:
> > > >     RetryError: resources.CephStorage: RetryError[<Future at
> > > > 0xdd62550
> > > >     state=finished returned bool>]
> > > >     2017-04-12 00:26:34.436989 | 2017-04-12 00:26:29Z
> > > >     [overcloud-CephStorage-bkucn6ign34i]:
> > > > UPDATE_FAILED  resources[0]:
> > > >     RetryError: resources.CephStorage: RetryError[<Future at
> > > > 0xdd62550
> > > >     state=finished returned bool>]
> > > >     2017-04-12 00:26:34.437078 | 2017-04-12 00:26:30Z
> > > >     [overcloud-Controller-3lf3jauv4cbc-0-
> > > > ydowkb3nwsso.Controller]:
> > > >     UPDATE_FAILED  RetryError: resources.Controller:
> > > > RetryError[<Future at
> > > >     0xdc79b50 state=finished returned bool>]
> > > >     2017-04-12 00:26:34.437173 | 2017-04-12 00:26:30Z
> > > >     [overcloud-Controller-3lf3jauv4cbc-0-ydowkb3nwsso]:
> > > > UPDATE_FAILED
> > > >     RetryError: resources.Controller: RetryError[<Future at
> > > > 0xdc79b50
> > > >     state=finished returned bool>]
> > > >     2017-04-12 00:26:34.437269 | 2017-04-12 00:26:30Z
> > > > [CephStorage]:
> > > >     UPDATE_FAILED  resources.CephStorage: resources[0]:
> > > > RetryError:
> > > >     resources.CephStorage: RetryError[<Future at 0xdd62550
> > > > state=finished
> > > >     returned bool>]
> > > >
> > > >     --
> > > >     Dan Sneddon         |  Senior Principal Software Engineer
> > > >     [hidden email] <mailto:[hidden email]> | 
> > > >     redhat.com/openstack <http://redhat.com/openstack>
> > > >     dsneddon:irc        |  @dxs:twitter
> > > >
> > > > -- 
> > > > Regards,
> > > > Rabi Misra
> > >
> > > Rabi,
> > >
> > > Thanks for the explanation on why the port replacement isn't
> > > working.
> > >
> > > Unfortunately, I tried the recommendation that Thomas Herve made
> > > about
> > > using conditionals, but that failed. It appears that you can't
> > > use a
> > > get_attr inside of a conditional statement, I get an error to
> > > that
> > > effect. I can use a get_param, but that doesn't help me check a
> > > value in
> > > a nested stack.
> >
> > Yes I noticed the same thing recently, which is a limitation of
> > conditions
> > that IMO we should look at fixing - everywhere else in HOT
> > templates
> > get_param and get_attr can be used interchangably.
> >
> > As a workaround could you move the conditional into the port
> > resource
> > nested stack?  E.g if you returned the list we pass to "networks"
> > for the
> > server resource from the new ControlPlanePort nested stack perhaps,
> > and put
> > whatever logic we need to generate it inside the port nested stack?
> >
> > Attributes from e.g *-role.yaml would then be parameters in the
> > nested
> > stack, so you could do the conditional as Thomas suggested, or
> > perhaps
> > it's even simpler if you can just return different data from the
> > ctlplane.yaml and ctlplane_from_pool.yaml templates?
> >
> > Steve
>
> Steve,
>
> I tried moving the generation of the network parameters for the role
> into the port. That way, I shouldn't need a conditional because I can
> select which template to use (with or without a fixed_ip). That
> didn't
> work either, it doesn't appear that I can pass a list as a parameter
> to
> the "networks" property of TripleO::Server.
>
> "controller-role.yaml":
>   Controller:
>     type: OS::TripleO::Server
>     metadata:
>       os-collect-config:
>         command: {get_param: ConfigCommand}
>     properties:
>       image: {get_param: controllerImage}
>       image_update_policy: {get_param: ImageUpdatePolicy}
>       flavor: {get_param: OvercloudControlFlavor}
>       key_name: {get_param: KeyName}
>       networks: {get_attr: [ControlPlanePort, port_info]}
>
> Then, the ControlPlanePort has this:
> "ctlplane.yaml":
> outputs:
>   port_info:
>     description: The newtork name to use for port
>     value:
>       - network: {get_param: ControlPlaneNetName}
>
> OR "ctlplane_from_pool.yaml" defines the same value as:
>
>   port_info:
>     description: The newtork name and fixed_ip to use for port
>     value:
>       - network: {get_param: ControlPlaneNetName}
>         fixed_ip: {get_param: [IPPool, {get_param:
> ControlPlaneNetName},
> {get_param: NodeIndex}]}
>
> Unfortunately, this fails. For example, when passing only the
> network:
>
> """
> GET call to orchestration for
> http://172.20.0.1:8004/v1/f6d7244fe69c43359c293d2c9edf0ce5/stacks/ove
> r
> cloud-ObjectStorageIpListMap-kwmthuphny55/1d8b2ef4-8223-412b-b32f-
> 6e00a9651478/resources
> used reques
> t id req-50c24b3f-2118-444b-878f-94e7d0a3b949
> overcloud.Controller.0.Controller:
>   resource_type: OS::TripleO::Server
>   physical_resource_id:
>   status: CREATE_FAILED
>   status_reason: |
>     BadRequest: resources.Controller: Bad networks format: network
> uuid
> is not in proper format (ctlplane) (HTTP 400) (Request-ID:
> req-e3da2bf7-91bb-4902-bdcc-0ab11d1e2e09)
> """
>
> Is there a way that I can pass a list between a nested resource
> (port)
> and the parent stack (the role in this case), so I can pass either a
> network or a network plus fixed IP? I'm not sure why this is failing,
> since simply passing ctlplane works in the original role definition:
>
>   Controller:
>     type: OS::TripleO::Server
>     metadata:
>       os-collect-config:
>         command: {get_param: ConfigCommand}
>     properties:
>       image: {get_param: controllerImage}
>       image_update_policy: {get_param: ImageUpdatePolicy}
>       flavor: {get_param: OvercloudControlFlavor}
>       key_name: {get_param: KeyName}
>       networks:
>         - network: ctlplane
>

I have been playing whit this quite a bit. And AFAICT from testing it
does not work to create a template that output a dict or a list and
pass that in as a network using get_attr.

----------------------------
heat_template_version: ocata

parameters:
  ControlPlaneNetName:
    description: Name of the Control Plane neutron network
    default: provider
    type: string
  ControlPlaneSubnet:
    description: Name of the Control Plane neutron subnet
    default: provider
    type: string

outputs:
  portinfo:
    value: 
      - {network: {get_param: ControlPlaneNetName}, subnet: {get_param:
ControlPlaneSubnet}}

----------------------------
heat_template_version: ocata

parameters:
  FixedIP:
    type: string
    default: None 

resources:

  ctlplane_port:
    type: OS::TripleO::Compute::Ports::ControlPlanePort

  my_instance:
    type: OS::Nova::Server
    properties:
      image: cirros
      flavor: m1.small
      networks: {get_attr: [ctlplane_port, portinfo]}


openstack stack create -t ctlplane_port_from_template.yaml stack -e
resource_registry.yaml


Results:
--------
# openstack stack resource show stack my_instance -c
resource_status_reason -f json
{
  "resource_status_reason": "NotFound: resources.my_instance: The
resource could not be found.<br /><br />\n\n\n\nNeutron server returns
request_ids: ['req-feb1aa2e-e600-47e1-bed8-df18c556ec27']"
}


# openstack stack resource show stack ctlplane_port -c attributes -f
json
{
  "attributes": {
    "portinfo": [
      {
        "subnet": "provider", 
        "network": "provider"
      }
    ]
  }
}


AFAICT the data I try to send in, is correct.


I have also tried to pass only the dict like this:
 output:   
   portinfo:
     value: {network: {get_param: ControlPlaneNetName}, subnet:
{get_param: ControlPlaneSubnet}}

   networks: 
     - {get_attr: [ctlplane_port, portinfo]}

In this case I get this error:
ERROR: One of the properties "network", "port" or "subnet" should be
set for the specified network of server "my_instance".





One way to make it work, is to hack the resource constraints.
If I remove the ip_addr constraint of fixed_ip I can do:

   networks:
     network: {get_attr: [CtlPlanePort, network]}
     subnet: {get_attr: [CtlPlanePort, subnet]}
     fixed_ip: {get_attr: [CtlPlanePort, fixed_ip]}

If I don't have "fixed_ip" as output in the ctlplane_port template, it
become "None" and the with the constraint removed this works.

Would it be an option to create "IPConstraint_or_None" custom
constraint? To allow either None or Valid IP address? (Unless we can
fix Heat to treat data it get's from get_attr properly...)



--
|Harald Jensås

__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: [hidden email]?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Loading...