OpenStack › Developer

[zun] Failed to create Container

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

8 messages Options

Rania Adouni

[zun] Failed to create Container

Hi ,

I am starting to use zun on openstack pike , and when I try to lunch container with cirros image , I get this reason : Docker internal error: 500 Server Error: Internal Server Error ("legacy plugin: Post http://192.168.1.10:23750/Plugin.Activate: dial tcp 192.168.1.10:23750: connect: connection refused").

you can find here the log on my compute node :https://pastebin.com/nZTiTZiV.

some help with this will be great .

Best Regards,

Rania Adouni

__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: [hidden email]?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Jacob Burckhardt

Re: [zun] Failed to create Container

Your error message says it cannot connect to port 23750. That's the kuryr-libnetwork default listen port. On 192.168.1.10, run:

lsof -iTCP -sTCP:LISTEN -P

On my compute node, it outputs:

COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME

…

kuryr-ser 1976 root 6u IPv4 30896 0t0 TCP localhost:23750 (LISTEN)

So, you can see that kuryr is listening on 23750. If lsof does not list a process listening on 23750, then check if kuryr is up by running:

systemctl status kuryr-libnetwork

You can also try restarting it by running:

systemctl restart kuryr-libnetwork

I installed that service by following:

https://docs.openstack.org/kuryr-libnetwork/queens/install/compute-install-ubuntu.html

That is the queens install guide which might help you since zun has no install guide in pike.

If you see it listening, you can try this command:

telnet 192.168.1.10 23750

If it fails to connect, then try:

telnet localhost 23750

If it works with localhost but not 192.168.1.10, then I think that means you need to tell docker to connect to localhost instead of 192.168.1.10.

Can you get the commands at the following web page to succeed?

https://docs.openstack.org/kuryr-libnetwork/queens/install/verify.html

Jacob Burckhardt

Senior Software Engineer

pdvWireless

From: Rania Adouni <[hidden email]>
Sent: Tuesday, November 20, 2018 2:02 PM
To: [hidden email]
Subject: [openstack-dev] [zun] Failed to create Container

Hi ,

you can find here the log on my compute node :https://pastebin.com/nZTiTZiV.

some help with this will be great .

Best Regards,

Rania Adouni

Re: [zun] Failed to create Container

hi ,

Thanks for your reply and yes the problem that kuryr-libnetwork is failed to start with the error :

● kuryr-libnetwork.service - Kuryr-libnetwork - Docker network plugin for Neutron

Loaded: loaded (/etc/systemd/system/kuryr-libnetwork.service; enabled; vendor preset: enabled)

Active: failed (Result: exit-code) since mer. 2018-11-21 01:48:48 CET; 287ms ago

Process: 13974 ExecStart=/usr/local/bin/kuryr-server --config-file /etc/kuryr/kuryr.conf (code=exited, status=1/FAILURE)

Main PID: 13974 (code=exited, status=1/FAILURE)

nov. 21 01:48:48 compute kuryr-server[13974]: 2018-11-21 01:48:48.542 13974 ERROR kuryr value = self._do_get(name, group, namespace)

nov. 21 01:48:48 compute kuryr-server[13974]: 2018-11-21 01:48:48.542 13974 ERROR kuryr File "/usr/lib/python2.7/dist-packages/oslo_config/cfg.py", line 2942, in _

nov. 21 01:48:48 compute kuryr-server[13974]: 2018-11-21 01:48:48.542 13974 ERROR kuryr info = self._get_opt_info(name, group)

nov. 21 01:48:48 compute kuryr-server[13974]: 2018-11-21 01:48:48.542 13974 ERROR kuryr File "/usr/lib/python2.7/dist-packages/oslo_config/cfg.py", line 3099, in _

nov. 21 01:48:48 compute kuryr-server[13974]: 2018-11-21 01:48:48.542 13974 ERROR kuryr raise NoSuchOptError(opt_name, group)

nov. 21 01:48:48 compute kuryr-server[13974]: 2018-11-21 01:48:48.542 13974 ERROR kuryr NoSuchOptError: no such option driver in group [binding]

nov. 21 01:48:48 compute kuryr-server[13974]: 2018-11-21 01:48:48.542 13974 ERROR kuryr

nov. 21 01:48:48 compute systemd[1]: kuryr-libnetwork.service: Main process exited, code=exited, status=1/FAILURE

nov. 21 01:48:48 compute systemd[1]: kuryr-libnetwork.service: Unit entered failed state.

nov. 21 01:48:48 compute systemd[1]: kuryr-libnetwork.service: Failed with result 'exit-code'.

and also the command lsof -iTCP -sTCP:LISTEN -P doesn't show the kuryr service :

COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME

dockerd 1144 root 5u IPv4 20120 0t0 TCP compute:2375 (LISTEN)

sshd 1288 root 3u IPv4 19532 0t0 TCP *:22 (LISTEN)

sshd 1288 root 4u IPv6 19534 0t0 TCP *:22 (LISTEN)

dnsmasq 1961 nobody 6u IPv4 23450 0t0 TCP 192.168.122.1:53 (LISTEN)

sshd 4163 adouni 9u IPv6 34950 0t0 TCP localhost:6010 (LISTEN)

sshd 4163 adouni 10u IPv4 34951 0t0 TCP localhost:6010 (LISTEN)

qemu-syst 4621 libvirt-qemu 18u IPv4 37121 0t0 TCP *:5900 (LISTEN)

!!!

Le mer. 21 nov. 2018 à 01:45, Jacob Burckhardt <[hidden email]> a écrit :

Your error message says it cannot connect to port 23750. That's the kuryr-libnetwork default listen port. On 192.168.1.10, run:

lsof -iTCP -sTCP:LISTEN -P

On my compute node, it outputs:

COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME

…

kuryr-ser 1976 root 6u IPv4 30896 0t0 TCP localhost:23750 (LISTEN)

So, you can see that kuryr is listening on 23750. If lsof does not list a process listening on 23750, then check if kuryr is up by running:

systemctl status kuryr-libnetwork

You can also try restarting it by running:

systemctl restart kuryr-libnetwork

I installed that service by following:

https://docs.openstack.org/kuryr-libnetwork/queens/install/compute-install-ubuntu.html

That is the queens install guide which might help you since zun has no install guide in pike.

If you see it listening, you can try this command:

telnet 192.168.1.10 23750

If it fails to connect, then try:

telnet localhost 23750

If it works with localhost but not 192.168.1.10, then I think that means you need to tell docker to connect to localhost instead of 192.168.1.10.

Can you get the commands at the following web page to succeed?

https://docs.openstack.org/kuryr-libnetwork/queens/install/verify.html

Jacob Burckhardt

Senior Software Engineer

From: Rania Adouni <[hidden email]>
Sent: Tuesday, November 20, 2018 2:02 PM
To: [hidden email]
Subject: [openstack-dev] [zun] Failed to create Container

Hi ,

I am starting to use zun on openstack pike , and when I try to lunch container with cirros image , I get this reason : Docker internal error: 500 Server Error: Internal Server Error ("legacy plugin: Post http://192.168.1.10:23750/Plugin.Activate: dial tcp 192.168.1.10:23750: connect: connection refused").

you can find here the log on my compute node :https://pastebin.com/nZTiTZiV.

some help with this will be great .

Best Regards,

Rania Adouni

__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@...?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

image003.jpg (3K) Download Attachment

Jacob Burckhardt

Re: [zun] Failed to create Container

The important part of the error message seems to be:

NoSuchOptError: no such option driver in group [binding]

In my /etc/kuryr/kuryr.conf file, all the text is commented out in the [binding] section. Some of the commented options have "driver" in the name. Since they are commented, I guess they have defaults. I suggest commenting any option that mentions "driver" and then restart kuryr-libnetwork.

-Jacob Burckhardt

From: Rania Adouni <[hidden email]>
Sent: Tuesday, November 20, 2018 4:58 PM
To: [hidden email]
Subject: Re: [openstack-dev] [zun] Failed to create Container

hi ,
Thanks for your reply and yes the problem that kuryr-libnetwork  is failed to start with the error :
● kuryr-libnetwork.service - Kuryr-libnetwork - Docker network plugin for Neutron
Loaded: loaded (/etc/systemd/system/kuryr-libnetwork.service; enabled; vendor preset: enabled)
Active: failed (Result: exit-code) since mer. 2018-11-21 01:48:48 CET; 287ms ago
Process: 13974 ExecStart=/usr/local/bin/kuryr-server --config-file /etc/kuryr/kuryr.conf (code=exited, status=1/FAILURE)
Main PID: 13974 (code=exited, status=1/FAILURE)

nov. 21 01:48:48 compute kuryr-server[13974]: 2018-11-21 01:48:48.542 13974 ERROR kuryr value = self._do_get(name, group, namespace)
nov. 21 01:48:48 compute kuryr-server[13974]: 2018-11-21 01:48:48.542 13974 ERROR kuryr File "/usr/lib/python2.7/dist-packages/oslo_config/cfg.py", line 2942, in _
nov. 21 01:48:48 compute kuryr-server[13974]: 2018-11-21 01:48:48.542 13974 ERROR kuryr info = self._get_opt_info(name, group)
nov. 21 01:48:48 compute kuryr-server[13974]: 2018-11-21 01:48:48.542 13974 ERROR kuryr File "/usr/lib/python2.7/dist-packages/oslo_config/cfg.py", line 3099, in _
nov. 21 01:48:48 compute kuryr-server[13974]: 2018-11-21 01:48:48.542 13974 ERROR kuryr raise NoSuchOptError(opt_name, group)
nov. 21 01:48:48 compute kuryr-server[13974]: 2018-11-21 01:48:48.542 13974 ERROR kuryr NoSuchOptError: no such option driver in group [binding]
nov. 21 01:48:48 compute kuryr-server[13974]: 2018-11-21 01:48:48.542 13974 ERROR kuryr
nov. 21 01:48:48 compute systemd[1]: kuryr-libnetwork.service: Main process exited, code=exited, status=1/FAILURE
nov. 21 01:48:48 compute systemd[1]: kuryr-libnetwork.service: Unit entered failed state.
nov. 21 01:48:48 compute systemd[1]: kuryr-libnetwork.service: Failed with result 'exit-code'.

and also the command lsof -iTCP -sTCP:LISTEN -P doesn't show the kuryr service :
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
dockerd 1144 root 5u IPv4 20120 0t0 TCP compute:2375 (LISTEN)
sshd 1288 root 3u IPv4 19532 0t0 TCP *:22 (LISTEN)
sshd 1288 root 4u IPv6 19534 0t0 TCP *:22 (LISTEN)
dnsmasq 1961 nobody 6u IPv4 23450 0t0 TCP http://192.168.122.1:53 (LISTEN)
sshd 4163 adouni 9u IPv6 34950 0t0 TCP localhost:6010 (LISTEN)
sshd 4163 adouni 10u IPv4 34951 0t0 TCP localhost:6010 (LISTEN)
qemu-syst 4621 libvirt-qemu 18u IPv4 37121 0t0 TCP *:5900 (LISTEN)

!!!

Le mer. 21 nov. 2018 à 01:45, Jacob Burckhardt <mailto:[hidden email]> a écrit :
Your error message says it cannot connect to port 23750. That's the kuryr-libnetwork default listen port. On 192.168.1.10, run:

lsof -iTCP -sTCP:LISTEN -P

On my compute node, it outputs:

COMMAND    PID        USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
…
kuryr-ser 1976        root    6u IPv4 30896      0t0 TCP localhost:23750 (LISTEN)

So, you can see that kuryr is listening on 23750. If lsof does not list a process listening on 23750, then check if kuryr is up by running:

systemctl status kuryr-libnetwork

You can also try restarting it by running:

systemctl restart kuryr-libnetwork

I installed that service by following:

https://docs.openstack.org/kuryr-libnetwork/queens/install/compute-install-ubuntu.html

That is the queens install guide which might help you since zun has no install guide in pike.

If you see it listening, you can try this command:

telnet 192.168.1.10 23750

If it fails to connect, then try:

telnet localhost 23750

If it works with localhost but not 192.168.1.10, then I think that means you need to tell docker to connect to localhost instead of 192.168.1.10.

Can you get the commands at the following web page to succeed?

https://docs.openstack.org/kuryr-libnetwork/queens/install/verify.html

Jacob Burckhardt
Senior Software Engineer

From: Rania Adouni <mailto:[hidden email]>
Sent: Tuesday, November 20, 2018 2:02 PM
To: mailto:[hidden email]
Subject: [openstack-dev] [zun] Failed to create Container

Hi ,

I am starting to use zun on openstack pike , and when I try to lunch container with cirros image , I get this reason : Docker internal error: 500 Server Error: Internal Server Error ("legacy plugin: Post http://192.168.1.10:23750/Plugin.Activate: dial tcp http://192.168.1.10:23750/: connect: connection refused").
you can find here the log on my compute node :https://pastebin.com/nZTiTZiV.
some help with this will be great .
Best Regards,
Rania Adouni

__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: http://OpenStack-dev-request@...?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: [hidden email]?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Hongbin Lu

Re: [zun] Failed to create Container

In reply to this post by Rania Adouni

Hi Rania,

The config option 'driver' was recently renamed, so I guess the problem is the version of kuryr-lib is not right. The Pike release of Zun is matched to kuryr-lib 0.6.0 so you might want to confirm the version of kuryr-lib.

$ sudo pip freeze | grep kuryr

If the version is not right, uninstall and re-install the kuryr-lib, then restart kuryr-libnetwork. Let me know if it still doesn't work.

Best regards,

Hongbin

On Tue, Nov 20, 2018 at 8:01 PM Rania Adouni <[hidden email]> wrote:

hi ,
Thanks for your reply and yes the problem that kuryr-libnetwork  is failed to start with the error :
● kuryr-libnetwork.service - Kuryr-libnetwork - Docker network plugin for Neutron
Loaded: loaded (/etc/systemd/system/kuryr-libnetwork.service; enabled; vendor preset: enabled)
Active: failed (Result: exit-code) since mer. 2018-11-21 01:48:48 CET; 287ms ago
Process: 13974 ExecStart=/usr/local/bin/kuryr-server --config-file /etc/kuryr/kuryr.conf (code=exited, status=1/FAILURE)
Main PID: 13974 (code=exited, status=1/FAILURE)

nov. 21 01:48:48 compute kuryr-server[13974]: 2018-11-21 01:48:48.542 13974 ERROR kuryr value = self._do_get(name, group, namespace)
nov. 21 01:48:48 compute kuryr-server[13974]: 2018-11-21 01:48:48.542 13974 ERROR kuryr File "/usr/lib/python2.7/dist-packages/oslo_config/cfg.py", line 2942, in _
nov. 21 01:48:48 compute kuryr-server[13974]: 2018-11-21 01:48:48.542 13974 ERROR kuryr info = self._get_opt_info(name, group)
nov. 21 01:48:48 compute kuryr-server[13974]: 2018-11-21 01:48:48.542 13974 ERROR kuryr File "/usr/lib/python2.7/dist-packages/oslo_config/cfg.py", line 3099, in _
nov. 21 01:48:48 compute kuryr-server[13974]: 2018-11-21 01:48:48.542 13974 ERROR kuryr raise NoSuchOptError(opt_name, group)
nov. 21 01:48:48 compute kuryr-server[13974]: 2018-11-21 01:48:48.542 13974 ERROR kuryr NoSuchOptError: no such option driver in group [binding]
nov. 21 01:48:48 compute kuryr-server[13974]: 2018-11-21 01:48:48.542 13974 ERROR kuryr
nov. 21 01:48:48 compute systemd[1]: kuryr-libnetwork.service: Main process exited, code=exited, status=1/FAILURE
nov. 21 01:48:48 compute systemd[1]: kuryr-libnetwork.service: Unit entered failed state.
nov. 21 01:48:48 compute systemd[1]: kuryr-libnetwork.service: Failed with result 'exit-code'.

and also the command lsof -iTCP -sTCP:LISTEN -P doesn't show the kuryr service :
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
dockerd 1144 root 5u IPv4 20120 0t0 TCP compute:2375 (LISTEN)
sshd 1288 root 3u IPv4 19532 0t0 TCP *:22 (LISTEN)
sshd 1288 root 4u IPv6 19534 0t0 TCP *:22 (LISTEN)
dnsmasq 1961 nobody 6u IPv4 23450 0t0 TCP 192.168.122.1:53 (LISTEN)
sshd 4163 adouni 9u IPv6 34950 0t0 TCP localhost:6010 (LISTEN)
sshd 4163 adouni 10u IPv4 34951 0t0 TCP localhost:6010 (LISTEN)
qemu-syst 4621 libvirt-qemu 18u IPv4 37121 0t0 TCP *:5900 (LISTEN)

!!!

Le mer. 21 nov. 2018 à 01:45, Jacob Burckhardt <[hidden email]> a écrit :

Your error message says it cannot connect to port 23750. That's the kuryr-libnetwork default listen port. On 192.168.1.10, run:

lsof -iTCP -sTCP:LISTEN -P

On my compute node, it outputs:

COMMAND    PID        USER   FD   TYPE DEVICE SIZE/OFF NODE NAME

…

kuryr-ser 1976        root    6u IPv4 30896      0t0 TCP localhost:23750 (LISTEN)

So, you can see that kuryr is listening on 23750. If lsof does not list a process listening on 23750, then check if kuryr is up by running:

systemctl status kuryr-libnetwork

You can also try restarting it by running:

systemctl restart kuryr-libnetwork

I installed that service by following:

https://docs.openstack.org/kuryr-libnetwork/queens/install/compute-install-ubuntu.html

That is the queens install guide which might help you since zun has no install guide in pike.

If you see it listening, you can try this command:

telnet 192.168.1.10 23750

If it fails to connect, then try:

telnet localhost 23750

If it works with localhost but not 192.168.1.10, then I think that means you need to tell docker to connect to localhost instead of 192.168.1.10.

Can you get the commands at the following web page to succeed?

https://docs.openstack.org/kuryr-libnetwork/queens/install/verify.html

Jacob Burckhardt

Senior Software Engineer

From: Rania Adouni <[hidden email]>
Sent: Tuesday, November 20, 2018 2:02 PM
To: [hidden email]
Subject: [openstack-dev] [zun] Failed to create Container

Hi ,

I am starting to use zun on openstack pike , and when I try to lunch container with cirros image , I get this reason : Docker internal error: 500 Server Error: Internal Server Error ("legacy plugin: Post http://192.168.1.10:23750/Plugin.Activate: dial tcp 192.168.1.10:23750: connect: connection refused").

you can find here the log on my compute node :https://pastebin.com/nZTiTZiV.

some help with this will be great .

Best Regards,

Rania Adouni

__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@...?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@...?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Rania Adouni

Re: [zun] Failed to create Container

Hi Hongbin,

Yes the version of kuryr-lib is :0.8.0 , So I think I should remove it !

I should remove only the kuryr-lib ! if you have some command they can help me to unisntall it , it will be nice to provide it .

Best Regards,

Rania

Le mer. 21 nov. 2018 à 02:27, Hongbin Lu <[hidden email]> a écrit :

Hi Rania,

The config option 'driver' was recently renamed, so I guess the problem is the version of kuryr-lib is not right. The Pike release of Zun is matched to kuryr-lib 0.6.0 so you might want to confirm the version of kuryr-lib.

$ sudo pip freeze | grep kuryr

If the version is not right, uninstall and re-install the kuryr-lib, then restart kuryr-libnetwork. Let me know if it still doesn't work.

Best regards,
Hongbin

On Tue, Nov 20, 2018 at 8:01 PM Rania Adouni <[hidden email]> wrote:
hi ,
Thanks for your reply and yes the problem that kuryr-libnetwork  is failed to start with the error :
● kuryr-libnetwork.service - Kuryr-libnetwork - Docker network plugin for Neutron
Loaded: loaded (/etc/systemd/system/kuryr-libnetwork.service; enabled; vendor preset: enabled)
Active: failed (Result: exit-code) since mer. 2018-11-21 01:48:48 CET; 287ms ago
Process: 13974 ExecStart=/usr/local/bin/kuryr-server --config-file /etc/kuryr/kuryr.conf (code=exited, status=1/FAILURE)
Main PID: 13974 (code=exited, status=1/FAILURE)

nov. 21 01:48:48 compute kuryr-server[13974]: 2018-11-21 01:48:48.542 13974 ERROR kuryr value = self._do_get(name, group, namespace)
nov. 21 01:48:48 compute kuryr-server[13974]: 2018-11-21 01:48:48.542 13974 ERROR kuryr File "/usr/lib/python2.7/dist-packages/oslo_config/cfg.py", line 2942, in _
nov. 21 01:48:48 compute kuryr-server[13974]: 2018-11-21 01:48:48.542 13974 ERROR kuryr info = self._get_opt_info(name, group)
nov. 21 01:48:48 compute kuryr-server[13974]: 2018-11-21 01:48:48.542 13974 ERROR kuryr File "/usr/lib/python2.7/dist-packages/oslo_config/cfg.py", line 3099, in _
nov. 21 01:48:48 compute kuryr-server[13974]: 2018-11-21 01:48:48.542 13974 ERROR kuryr raise NoSuchOptError(opt_name, group)
nov. 21 01:48:48 compute kuryr-server[13974]: 2018-11-21 01:48:48.542 13974 ERROR kuryr NoSuchOptError: no such option driver in group [binding]
nov. 21 01:48:48 compute kuryr-server[13974]: 2018-11-21 01:48:48.542 13974 ERROR kuryr
nov. 21 01:48:48 compute systemd[1]: kuryr-libnetwork.service: Main process exited, code=exited, status=1/FAILURE
nov. 21 01:48:48 compute systemd[1]: kuryr-libnetwork.service: Unit entered failed state.
nov. 21 01:48:48 compute systemd[1]: kuryr-libnetwork.service: Failed with result 'exit-code'.

and also the command lsof -iTCP -sTCP:LISTEN -P doesn't show the kuryr service :
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
dockerd 1144 root 5u IPv4 20120 0t0 TCP compute:2375 (LISTEN)
sshd 1288 root 3u IPv4 19532 0t0 TCP *:22 (LISTEN)
sshd 1288 root 4u IPv6 19534 0t0 TCP *:22 (LISTEN)
dnsmasq 1961 nobody 6u IPv4 23450 0t0 TCP 192.168.122.1:53 (LISTEN)
sshd 4163 adouni 9u IPv6 34950 0t0 TCP localhost:6010 (LISTEN)
sshd 4163 adouni 10u IPv4 34951 0t0 TCP localhost:6010 (LISTEN)
qemu-syst 4621 libvirt-qemu 18u IPv4 37121 0t0 TCP *:5900 (LISTEN)

!!!

Le mer. 21 nov. 2018 à 01:45, Jacob Burckhardt <[hidden email]> a écrit :

Your error message says it cannot connect to port 23750. That's the kuryr-libnetwork default listen port. On 192.168.1.10, run:

lsof -iTCP -sTCP:LISTEN -P

On my compute node, it outputs:

COMMAND    PID        USER   FD   TYPE DEVICE SIZE/OFF NODE NAME

…

kuryr-ser 1976        root    6u IPv4 30896      0t0 TCP localhost:23750 (LISTEN)

So, you can see that kuryr is listening on 23750. If lsof does not list a process listening on 23750, then check if kuryr is up by running:

systemctl status kuryr-libnetwork

You can also try restarting it by running:

systemctl restart kuryr-libnetwork

I installed that service by following:

https://docs.openstack.org/kuryr-libnetwork/queens/install/compute-install-ubuntu.html

That is the queens install guide which might help you since zun has no install guide in pike.

If you see it listening, you can try this command:

telnet 192.168.1.10 23750

If it fails to connect, then try:

telnet localhost 23750

If it works with localhost but not 192.168.1.10, then I think that means you need to tell docker to connect to localhost instead of 192.168.1.10.

Can you get the commands at the following web page to succeed?

https://docs.openstack.org/kuryr-libnetwork/queens/install/verify.html

Jacob Burckhardt

Senior Software Engineer

From: Rania Adouni <[hidden email]>
Sent: Tuesday, November 20, 2018 2:02 PM
To: [hidden email]
Subject: [openstack-dev] [zun] Failed to create Container

Hi ,

I am starting to use zun on openstack pike , and when I try to lunch container with cirros image , I get this reason : Docker internal error: 500 Server Error: Internal Server Error ("legacy plugin: Post http://192.168.1.10:23750/Plugin.Activate: dial tcp 192.168.1.10:23750: connect: connection refused").

you can find here the log on my compute node :https://pastebin.com/nZTiTZiV.

some help with this will be great .

Best Regards,

Rania Adouni

__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@...?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@...?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@...?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Rania Adouni

Re: [zun] Failed to create Container

Hi Hongbin ,

thanks for your advice I did fix it with changing the kuryr-lib to 0.6.0 and my service just work fine and also with lsof -iTCP -sTCP:LISTEN -P

I can see it in the list :

kuryr-ser 16581 root 5u IPv4 91929 0t0 TCP localhost:23750 (LISTEN) ------------> that is great.

+ but I have two questions :

1- How I can change the localhost to my compute address (192.168.1.10:23750) because it is still the same error when trying to create container " Docker internal error: 500 Server Error: Internal Server Error ("legacy plugin: Post http://192.168.1.10:23750/Plugin.Activate: dial tcp 192.168.1.10:23750: connect: connection refused"). ?

2- my other question is about the log of zun why I can find it under my directory /var/log ?

Best Regards,

Rania Adouni

Le mer. 21 nov. 2018 à 02:34, Rania Adouni <[hidden email]> a écrit :

Hi Hongbin,
Yes the version of kuryr-lib is :0.8.0 , So I think I should remove it !
I should remove only the kuryr-lib ! if you have some command they can help me to unisntall it , it will be nice to provide it .

Best Regards,
Rania

Le mer. 21 nov. 2018 à 02:27, Hongbin Lu <[hidden email]> a écrit :
Hi Rania,

The config option 'driver' was recently renamed, so I guess the problem is the version of kuryr-lib is not right. The Pike release of Zun is matched to kuryr-lib 0.6.0 so you might want to confirm the version of kuryr-lib.

$ sudo pip freeze | grep kuryr

If the version is not right, uninstall and re-install the kuryr-lib, then restart kuryr-libnetwork. Let me know if it still doesn't work.

Best regards,
Hongbin

On Tue, Nov 20, 2018 at 8:01 PM Rania Adouni <[hidden email]> wrote:
hi ,
Thanks for your reply and yes the problem that kuryr-libnetwork  is failed to start with the error :
● kuryr-libnetwork.service - Kuryr-libnetwork - Docker network plugin for Neutron
Loaded: loaded (/etc/systemd/system/kuryr-libnetwork.service; enabled; vendor preset: enabled)
Active: failed (Result: exit-code) since mer. 2018-11-21 01:48:48 CET; 287ms ago
Process: 13974 ExecStart=/usr/local/bin/kuryr-server --config-file /etc/kuryr/kuryr.conf (code=exited, status=1/FAILURE)
Main PID: 13974 (code=exited, status=1/FAILURE)

nov. 21 01:48:48 compute kuryr-server[13974]: 2018-11-21 01:48:48.542 13974 ERROR kuryr value = self._do_get(name, group, namespace)
nov. 21 01:48:48 compute kuryr-server[13974]: 2018-11-21 01:48:48.542 13974 ERROR kuryr File "/usr/lib/python2.7/dist-packages/oslo_config/cfg.py", line 2942, in _
nov. 21 01:48:48 compute kuryr-server[13974]: 2018-11-21 01:48:48.542 13974 ERROR kuryr info = self._get_opt_info(name, group)
nov. 21 01:48:48 compute kuryr-server[13974]: 2018-11-21 01:48:48.542 13974 ERROR kuryr File "/usr/lib/python2.7/dist-packages/oslo_config/cfg.py", line 3099, in _
nov. 21 01:48:48 compute kuryr-server[13974]: 2018-11-21 01:48:48.542 13974 ERROR kuryr raise NoSuchOptError(opt_name, group)
nov. 21 01:48:48 compute kuryr-server[13974]: 2018-11-21 01:48:48.542 13974 ERROR kuryr NoSuchOptError: no such option driver in group [binding]
nov. 21 01:48:48 compute kuryr-server[13974]: 2018-11-21 01:48:48.542 13974 ERROR kuryr
nov. 21 01:48:48 compute systemd[1]: kuryr-libnetwork.service: Main process exited, code=exited, status=1/FAILURE
nov. 21 01:48:48 compute systemd[1]: kuryr-libnetwork.service: Unit entered failed state.
nov. 21 01:48:48 compute systemd[1]: kuryr-libnetwork.service: Failed with result 'exit-code'.

and also the command lsof -iTCP -sTCP:LISTEN -P doesn't show the kuryr service :
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
dockerd 1144 root 5u IPv4 20120 0t0 TCP compute:2375 (LISTEN)
sshd 1288 root 3u IPv4 19532 0t0 TCP *:22 (LISTEN)
sshd 1288 root 4u IPv6 19534 0t0 TCP *:22 (LISTEN)
dnsmasq 1961 nobody 6u IPv4 23450 0t0 TCP 192.168.122.1:53 (LISTEN)
sshd 4163 adouni 9u IPv6 34950 0t0 TCP localhost:6010 (LISTEN)
sshd 4163 adouni 10u IPv4 34951 0t0 TCP localhost:6010 (LISTEN)
qemu-syst 4621 libvirt-qemu 18u IPv4 37121 0t0 TCP *:5900 (LISTEN)

!!!

Le mer. 21 nov. 2018 à 01:45, Jacob Burckhardt <[hidden email]> a écrit :

Your error message says it cannot connect to port 23750. That's the kuryr-libnetwork default listen port. On 192.168.1.10, run:

lsof -iTCP -sTCP:LISTEN -P

On my compute node, it outputs:

COMMAND    PID        USER   FD   TYPE DEVICE SIZE/OFF NODE NAME

…

kuryr-ser 1976        root    6u IPv4 30896      0t0 TCP localhost:23750 (LISTEN)

So, you can see that kuryr is listening on 23750. If lsof does not list a process listening on 23750, then check if kuryr is up by running:

systemctl status kuryr-libnetwork

You can also try restarting it by running:

systemctl restart kuryr-libnetwork

I installed that service by following:

https://docs.openstack.org/kuryr-libnetwork/queens/install/compute-install-ubuntu.html

That is the queens install guide which might help you since zun has no install guide in pike.

If you see it listening, you can try this command:

telnet 192.168.1.10 23750

If it fails to connect, then try:

telnet localhost 23750

If it works with localhost but not 192.168.1.10, then I think that means you need to tell docker to connect to localhost instead of 192.168.1.10.

Can you get the commands at the following web page to succeed?

https://docs.openstack.org/kuryr-libnetwork/queens/install/verify.html

Jacob Burckhardt

Senior Software Engineer

From: Rania Adouni <[hidden email]>
Sent: Tuesday, November 20, 2018 2:02 PM
To: [hidden email]
Subject: [openstack-dev] [zun] Failed to create Container

Hi ,

I am starting to use zun on openstack pike , and when I try to lunch container with cirros image , I get this reason : Docker internal error: 500 Server Error: Internal Server Error ("legacy plugin: Post http://192.168.1.10:23750/Plugin.Activate: dial tcp 192.168.1.10:23750: connect: connection refused").

you can find here the log on my compute node :https://pastebin.com/nZTiTZiV.

some help with this will be great .

Best Regards,

Rania Adouni

__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@...?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@...?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@...?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Hongbin Lu

Re: [zun] Failed to create Container

Rania,

For question #1, you can do the followings in *each compute node*:

* In your kuryr config file (i.e. /etc/kuryr/kuryr.conf), change the config [DEFAULT]kuryr_uri . For example:

[DEFAULT]

kuryr_uri = http://192.168.1.10:23750

This config decides the IP address and port that the kuryr-libnetwork process will listen to. By default, it listens to localhost which should be good enough for most of the cases. Therefore, change it only if it is necessary.

* In the file: /usr/lib/docker/plugins/kuryr/kuryr.spec , change the URL so that docker will know how to connect to kuryr-libnetwork. The value must match the kuryr_uri config above. Again, change it only if it is necessary.

Then, restart the docker and kuryr-libnetwork process. Let me know if it still doesn't work for you.

For question #2, I remembered all processes are running in systemd, so you can retrieve logs by using journalctl. For example:

$ journalctl -u kuryr-libnetwork > kuryr.log

$ journalctl -u zun-compute > zun-compute.log

$ journalctl -u zun-api > zun-api.log

Best regards,

Hongbin

On Tue, Nov 20, 2018 at 9:01 PM Rania Adouni <[hidden email]> wrote:

Hi Hongbin ,
thanks for your advice I did fix it with changing the kuryr-lib to 0.6.0 and my service just work fine and also with lsof -iTCP -sTCP:LISTEN -P
I can see it in the list :
kuryr-ser 16581 root 5u IPv4 91929 0t0 TCP localhost:23750 (LISTEN) ------------> that is great.

+ but I have two questions :
1- How I can change the localhost to my compute address (192.168.1.10:23750) because it is still the same error when trying to create container " Docker internal error: 500 Server Error: Internal Server Error ("legacy plugin: Post http://192.168.1.10:23750/Plugin.Activate: dial tcp 192.168.1.10:23750: connect: connection refused"). ?

2- my other question is about the log of zun why I can find it under my directory /var/log ?

Best Regards,
Rania Adouni

Le mer. 21 nov. 2018 à 02:34, Rania Adouni <[hidden email]> a écrit :
Hi Hongbin,
Yes the version of kuryr-lib is :0.8.0 , So I think I should remove it !
I should remove only the kuryr-lib ! if you have some command they can help me to unisntall it , it will be nice to provide it .

Best Regards,
Rania

Le mer. 21 nov. 2018 à 02:27, Hongbin Lu <[hidden email]> a écrit :
Hi Rania,

The config option 'driver' was recently renamed, so I guess the problem is the version of kuryr-lib is not right. The Pike release of Zun is matched to kuryr-lib 0.6.0 so you might want to confirm the version of kuryr-lib.

$ sudo pip freeze | grep kuryr

If the version is not right, uninstall and re-install the kuryr-lib, then restart kuryr-libnetwork. Let me know if it still doesn't work.

Best regards,
Hongbin

On Tue, Nov 20, 2018 at 8:01 PM Rania Adouni <[hidden email]> wrote:
hi ,
Thanks for your reply and yes the problem that kuryr-libnetwork  is failed to start with the error :
● kuryr-libnetwork.service - Kuryr-libnetwork - Docker network plugin for Neutron
Loaded: loaded (/etc/systemd/system/kuryr-libnetwork.service; enabled; vendor preset: enabled)
Active: failed (Result: exit-code) since mer. 2018-11-21 01:48:48 CET; 287ms ago
Process: 13974 ExecStart=/usr/local/bin/kuryr-server --config-file /etc/kuryr/kuryr.conf (code=exited, status=1/FAILURE)
Main PID: 13974 (code=exited, status=1/FAILURE)

nov. 21 01:48:48 compute kuryr-server[13974]: 2018-11-21 01:48:48.542 13974 ERROR kuryr value = self._do_get(name, group, namespace)
nov. 21 01:48:48 compute kuryr-server[13974]: 2018-11-21 01:48:48.542 13974 ERROR kuryr File "/usr/lib/python2.7/dist-packages/oslo_config/cfg.py", line 2942, in _
nov. 21 01:48:48 compute kuryr-server[13974]: 2018-11-21 01:48:48.542 13974 ERROR kuryr info = self._get_opt_info(name, group)
nov. 21 01:48:48 compute kuryr-server[13974]: 2018-11-21 01:48:48.542 13974 ERROR kuryr File "/usr/lib/python2.7/dist-packages/oslo_config/cfg.py", line 3099, in _
nov. 21 01:48:48 compute kuryr-server[13974]: 2018-11-21 01:48:48.542 13974 ERROR kuryr raise NoSuchOptError(opt_name, group)
nov. 21 01:48:48 compute kuryr-server[13974]: 2018-11-21 01:48:48.542 13974 ERROR kuryr NoSuchOptError: no such option driver in group [binding]
nov. 21 01:48:48 compute kuryr-server[13974]: 2018-11-21 01:48:48.542 13974 ERROR kuryr
nov. 21 01:48:48 compute systemd[1]: kuryr-libnetwork.service: Main process exited, code=exited, status=1/FAILURE
nov. 21 01:48:48 compute systemd[1]: kuryr-libnetwork.service: Unit entered failed state.
nov. 21 01:48:48 compute systemd[1]: kuryr-libnetwork.service: Failed with result 'exit-code'.

and also the command lsof -iTCP -sTCP:LISTEN -P doesn't show the kuryr service :
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
dockerd 1144 root 5u IPv4 20120 0t0 TCP compute:2375 (LISTEN)
sshd 1288 root 3u IPv4 19532 0t0 TCP *:22 (LISTEN)
sshd 1288 root 4u IPv6 19534 0t0 TCP *:22 (LISTEN)
dnsmasq 1961 nobody 6u IPv4 23450 0t0 TCP 192.168.122.1:53 (LISTEN)
sshd 4163 adouni 9u IPv6 34950 0t0 TCP localhost:6010 (LISTEN)
sshd 4163 adouni 10u IPv4 34951 0t0 TCP localhost:6010 (LISTEN)
qemu-syst 4621 libvirt-qemu 18u IPv4 37121 0t0 TCP *:5900 (LISTEN)

!!!

Le mer. 21 nov. 2018 à 01:45, Jacob Burckhardt <[hidden email]> a écrit :

Your error message says it cannot connect to port 23750. That's the kuryr-libnetwork default listen port. On 192.168.1.10, run:

lsof -iTCP -sTCP:LISTEN -P

On my compute node, it outputs:

COMMAND    PID        USER   FD   TYPE DEVICE SIZE/OFF NODE NAME

…

kuryr-ser 1976        root    6u IPv4 30896      0t0 TCP localhost:23750 (LISTEN)

So, you can see that kuryr is listening on 23750. If lsof does not list a process listening on 23750, then check if kuryr is up by running:

systemctl status kuryr-libnetwork

You can also try restarting it by running:

systemctl restart kuryr-libnetwork

I installed that service by following:

https://docs.openstack.org/kuryr-libnetwork/queens/install/compute-install-ubuntu.html

That is the queens install guide which might help you since zun has no install guide in pike.

If you see it listening, you can try this command:

telnet 192.168.1.10 23750

If it fails to connect, then try:

telnet localhost 23750

If it works with localhost but not 192.168.1.10, then I think that means you need to tell docker to connect to localhost instead of 192.168.1.10.

Can you get the commands at the following web page to succeed?

https://docs.openstack.org/kuryr-libnetwork/queens/install/verify.html

Jacob Burckhardt

Senior Software Engineer

From: Rania Adouni <[hidden email]>
Sent: Tuesday, November 20, 2018 2:02 PM
To: [hidden email]
Subject: [openstack-dev] [zun] Failed to create Container

Hi ,

I am starting to use zun on openstack pike , and when I try to lunch container with cirros image , I get this reason : Docker internal error: 500 Server Error: Internal Server Error ("legacy plugin: Post http://192.168.1.10:23750/Plugin.Activate: dial tcp 192.168.1.10:23750: connect: connection refused").

you can find here the log on my compute node :https://pastebin.com/nZTiTZiV.

some help with this will be great .

Best Regards,

Rania Adouni

__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@...?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@...?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@...?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@...?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev