Problem running Openstack on fedora 16 (nova segfault)

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Problem running Openstack on fedora 16 (nova segfault)

Sergey Kulanov
Hi,
I have the following troubles while using Openstack on Fedora 16:
SOFTWARE (what do we have):
- Fedora 16 Beta
- Linux server1.example.com 3.1.0-0.rc9.git0.0.fc16.i686.PAE #1 SMP Wed
Oct 5 15:51:55 UTC 2011 i686 i686 i386 GNU/Linux
-   openstack-swift-auth-1.4.0-2.fc16.noarch
     openstack-glance-2011.3-1.fc16.noarch
     openstack-swift-1.4.0-2.fc16.noarch
     openstack-swift-proxy-1.4.0-2.fc16.noarch
     openstack-swift-account-1.4.0-2.fc16.noarch
     openstack-nova-2011.3-3.fc16.noarch
     openstack-swift-object-1.4.0-2.fc16.noarch
     openstack-swift-container-1.4.0-2.fc16.noarch
- glibc-2.14.90-11
-python-2.7.2-4.fc16.i686

I tried to follow this instruction
http://fedoraproject.org/wiki/Getting_started_with_OpenStack_Nova

INSTALLATION:
1) Some installation warnings:
Downloading Packages:
(1/5):
openstack-swift-account-1.4.0-2.fc16.noarch.rpm                                                          
|  26 kB     00:00
(2/5):
openstack-swift-auth-1.4.0-2.fc16.noarch.rpm                                                              
| 9.9 kB     00:00
(3/5):
openstack-swift-container-1.4.0-2.fc16.noarch.rpm                                                        
|  26 kB     00:00
(4/5):
openstack-swift-object-1.4.0-2.fc16.noarch.rpm                                                            
|  44 kB     00:00
(5/5):
openstack-swift-proxy-1.4.0-2.fc16.noarch.rpm                                                            
|  37 kB     00:00
----------------------------------------------------------------------------------------------------------------------------------------
Total                                                                                                  
137 kB/s | 143 kB     00:01
Running Transaction Check
Running Transaction Test
Transaction Test Succeeded
Running Transaction
   Installing :
openstack-swift-object-1.4.0-2.fc16.noarch                                                                          
1/5
Non-fatal POSTIN scriptlet failure in rpm package
openstack-swift-object-1.4.0-2.fc16.noarch
error reading information on service swift-object: No such file or directory
warning: %post(openstack-swift-object-1.4.0-2.fc16.noarch) scriptlet
failed, exit status 1
   Installing :
openstack-swift-proxy-1.4.0-2.fc16.noarch                                                                            
2/5
Non-fatal POSTIN scriptlet failure in rpm package
openstack-swift-proxy-1.4.0-2.fc16.noarch
error reading information on service swift-proxy: No such file or directory
warning: %post(openstack-swift-proxy-1.4.0-2.fc16.noarch) scriptlet
failed, exit status 1
   Installing :
openstack-swift-auth-1.4.0-2.fc16.noarch                                                                            
3/5
Non-fatal POSTIN scriptlet failure in rpm package
openstack-swift-auth-1.4.0-2.fc16.noarch
error reading information on service swift-auth: No such file or directory
warning: %post(openstack-swift-auth-1.4.0-2.fc16.noarch) scriptlet
failed, exit status 1
   Installing :
openstack-swift-account-1.4.0-2.fc16.noarch                                                                          
4/5
Non-fatal POSTIN scriptlet failure in rpm package
openstack-swift-account-1.4.0-2.fc16.noarch
error reading information on service swift-account: No such file or
directory
warning: %post(openstack-swift-account-1.4.0-2.fc16.noarch) scriptlet
failed, exit status 1
   Installing :
openstack-swift-container-1.4.0-2.fc16.noarch                                                                        
5/5
Non-fatal POSTIN scriptlet failure in rpm package
openstack-swift-container-1.4.0-2.fc16.noarch
error reading information on service swift-container: No such file or
directory
warning: %post(openstack-swift-container-1.4.0-2.fc16.noarch) scriptlet
failed, exit status 1

Installed:
   openstack-swift-account.noarch 0:1.4.0-2.fc16                        
openstack-swift-auth.noarch 0:1.4.0-2.fc16
   openstack-swift-container.noarch 0:1.4.0-2.fc16                      
openstack-swift-object.noarch 0:1.4.0-2.fc16
   openstack-swift-proxy.noarch 0:1.4.0-2.fc16

Complete!

2) RUNNING: I didn't change any default setting (just add debugging flag)
     [root at server1 ~]# service openstack-glance-api start; service
openstack-glance-registry start
     /var/log/messages
     Oct 14 21:20:31 server1 glance-api[1404]: Traceback (most recent
call last):
     Oct 14 21:20:31 server1 glance-api[1404]: File
"/usr/lib/python2.7/site-packages/eventlet/hubs/hub.py", line 336, in
fire_timers
     Oct 14 21:20:31 server1 glance-api[1404]: timer()
     Oct 14 21:20:31 server1 glance-api[1404]: File
"/usr/lib/python2.7/site-packages/eventlet/hubs/timer.py", line 56, in
__call__
     Oct 14 21:20:31 server1 glance-api[1404]: cb(*args, **kw)
     Oct 14 21:20:31 server1 glance-api[1404]: SystemError: error return
without exception set
     Oct 14 21:25:37 server1 glance-registry[1460]: Traceback (most
recent call last):
     Oct 14 21:25:37 server1 glance-registry[1460]: File
"/usr/lib/python2.7/site-packages/eventlet/hubs/hub.py", line 336, in
fire_timers
     Oct 14 21:25:37 server1 glance-registry[1460]: timer()
     Oct 14 21:25:37 server1 glance-registry[1460]: File
"/usr/lib/python2.7/site-packages/eventlet/hubs/timer.py", line 56, in
__call__
     Oct 14 21:25:37 server1 glance-registry[1460]: cb(*args, **kw)
     Oct 14 21:25:37 server1 glance-registry[1460]: SystemError: error
return without exception set

[root at server1 ~]# service openstack-glance-api status;service
openstack-glance-registry status
Redirecting to /bin/systemctl  status openstack-glance-api.service

           Loaded: loaded
(/lib/systemd/system/openstack-glance-api.service; disabled)
           Active: active (running) since Fri, 14 Oct 2011 21:20:28
+0300; 6min ago
         Main PID: 1404 (glance-api)
           CGroup: name=systemd:/system/openstack-glance-api.service
                   ? 1404 /usr/bin/python /usr/bin/glance-api
--config-file /etc/glance/glance-api.conf

openstack-glance-registry.service - OpenStack Image Service (code-named
Glance) Registry server
           Loaded: loaded
(/lib/systemd/system/openstack-glance-registry.service; disabled)
           Active: active (running) since Fri, 14 Oct 2011 21:25:37
+0300; 1min 29s ago
         Main PID: 1460 (glance-registry)
           CGroup: name=systemd:/system/openstack-glance-registry.service
                   ? 1460 /usr/bin/python /usr/bin/glance-registry
--config-file /etc/glance/glance-registry.conf

---------------------------NOVA START -----------------------
service openstack-nova-api start

/var/log/messages
Oct 14 21:30:55 server1 kernel: [ 1186.188224] nova-api[1560]: segfault
at 4 ip 0025a950 sp bfef9108 error 4 in libc-2.14.90.so[115000+1a7000]
Oct 14 21:30:55 server1 systemd[1]: openstack-nova-api.service: main
process exited, code=killed, status=11
Oct 14 21:30:55 server1 systemd[1]: Unit openstack-nova-api.service
entered failed state.

/var/log/nova/api.log
2011-10-14 21:30:54,785 INFO sqlalchemy.orm.mapper.Mapper [-]
(Image|images) _configure_property(created_at, Column)
2011-10-14 21:30:54,804 INFO sqlalchemy.orm.mapper.Mapper [-]
(Image|images) _configure_property(updated_at, Column)
2011-10-14 21:30:54,805 INFO sqlalchemy.orm.mapper.Mapper [-]
(Image|images) _configure_property(deleted_at, Column)
2011-10-14 21:30:54,805 INFO sqlalchemy.orm.mapper.Mapper [-]
(Image|images) _configure_property(deleted, Column)
2011-10-14 21:30:54,805 INFO sqlalchemy.orm.mapper.Mapper [-]
(Image|images) _configure_property(id, Column)
2011-10-14 21:30:54,805 INFO sqlalchemy.orm.mapper.Mapper [-]
(Image|images) _configure_property(name, Column)
2011-10-14 21:30:54,805 INFO sqlalchemy.orm.mapper.Mapper [-]
(Image|images) _configure_property(disk_format, Column)
2011-10-14 21:30:54,806 INFO sqlalchemy.orm.mapper.Mapper [-]
(Image|images) _configure_property(container_format, Column)
2011-10-14 21:30:54,806 INFO sqlalchemy.orm.mapper.Mapper [-]
(Image|images) _configure_property(size, Column)
2011-10-14 21:30:54,807 INFO sqlalchemy.orm.mapper.Mapper [-]
(Image|images) _configure_property(status, Column)
2011-10-14 21:30:54,807 INFO sqlalchemy.orm.mapper.Mapper [-]
(Image|images) _configure_property(is_public, Column)
2011-10-14 21:30:54,807 INFO sqlalchemy.orm.mapper.Mapper [-]
(Image|images) _configure_property(location, Column)
2011-10-14 21:30:54,807 INFO sqlalchemy.orm.mapper.Mapper [-]
(Image|images) _configure_property(checksum, Column)
2011-10-14 21:30:54,807 INFO sqlalchemy.orm.mapper.Mapper [-]
(Image|images) _configure_property(min_disk, Column)
2011-10-14 21:30:54,808 INFO sqlalchemy.orm.mapper.Mapper [-]
(Image|images) _configure_property(min_ram, Column)
2011-10-14 21:30:54,808 INFO sqlalchemy.orm.mapper.Mapper [-]
(Image|images) _configure_property(owner, Column)
2011-10-14 21:30:54,808 INFO sqlalchemy.orm.mapper.Mapper [-]
(Image|images) Identified primary key columns: ColumnSet([Column('id',
Integer(), table=<images>, primary_key=True, nullable=False)])
2011-10-14 21:30:54,809 INFO sqlalchemy.orm.mapper.Mapper [-]
(Image|images) constructed
2011-10-14 21:30:54,811 INFO sqlalchemy.orm.mapper.Mapper [-]
(ImageProperty|image_properties) _configure_property(image,
RelationshipProperty)
2011-10-14 21:30:54,812 INFO sqlalchemy.orm.mapper.Mapper [-]
(ImageProperty|image_properties) _configure_property(created_at, Column)
2011-10-14 21:30:54,812 INFO sqlalchemy.orm.mapper.Mapper [-]
(ImageProperty|image_properties) _configure_property(updated_at, Column)
2011-10-14 21:30:54,812 INFO sqlalchemy.orm.mapper.Mapper [-]
(ImageProperty|image_properties) _configure_property(deleted_at, Column)
2011-10-14 21:30:54,812 INFO sqlalchemy.orm.mapper.Mapper [-]
(ImageProperty|image_properties) _configure_property(deleted, Column)
2011-10-14 21:30:54,813 INFO sqlalchemy.orm.mapper.Mapper [-]
(ImageProperty|image_properties) _configure_property(id, Column)
2011-10-14 21:30:54,813 INFO sqlalchemy.orm.mapper.Mapper [-]
(ImageProperty|image_properties) _configure_property(image_id, Column)
2011-10-14 21:30:54,813 INFO sqlalchemy.orm.mapper.Mapper [-]
(ImageProperty|image_properties) _configure_property(name, Column)
2011-10-14 21:30:54,813 INFO sqlalchemy.orm.mapper.Mapper [-]
(ImageProperty|image_properties) _configure_property(value, Column)
2011-10-14 21:30:54,814 INFO sqlalchemy.orm.mapper.Mapper [-]
(ImageProperty|image_properties) Identified primary key columns:
ColumnSet([Column('id', Integer(), table=<image_properties>,
primary_key=True, nullable=False)])
2011-10-14 21:30:54,814 INFO sqlalchemy.orm.mapper.Mapper [-]
(ImageProperty|image_properties) constructed
2011-10-14 21:30:54,816 INFO sqlalchemy.orm.mapper.Mapper [-]
(ImageMember|image_members) _configure_property(image, RelationshipProperty)
2011-10-14 21:30:54,817 INFO sqlalchemy.orm.mapper.Mapper [-]
(ImageMember|image_members) _configure_property(created_at, Column)
2011-10-14 21:30:54,817 INFO sqlalchemy.orm.mapper.Mapper [-]
(ImageMember|image_members) _configure_property(updated_at, Column)
2011-10-14 21:30:54,817 INFO sqlalchemy.orm.mapper.Mapper [-]
(ImageMember|image_members) _configure_property(deleted_at, Column)
2011-10-14 21:30:54,817 INFO sqlalchemy.orm.mapper.Mapper [-]
(ImageMember|image_members) _configure_property(deleted, Column)
2011-10-14 21:30:54,818 INFO sqlalchemy.orm.mapper.Mapper [-]
(ImageMember|image_members) _configure_property(id, Column)
2011-10-14 21:30:54,818 INFO sqlalchemy.orm.mapper.Mapper [-]
(ImageMember|image_members) _configure_property(image_id, Column)
2011-10-14 21:30:54,818 INFO sqlalchemy.orm.mapper.Mapper [-]
(ImageMember|image_members) _configure_property(member, Column)
2011-10-14 21:30:54,818 INFO sqlalchemy.orm.mapper.Mapper [-]
(ImageMember|image_members) _configure_property(can_share, Column)
2011-10-14 21:30:54,819 INFO sqlalchemy.orm.mapper.Mapper [-]
(ImageMember|image_members) Identified primary key columns:
ColumnSet([Column('id', Integer(), table=<image_members>,
primary_key=True, nullable=False)])
2011-10-14 21:30:54,819 INFO sqlalchemy.orm.mapper.Mapper [-]
(ImageMember|image_members) constructed
2011-10-14 21:30:55,188 DEBUG nova.utils [-] Running sh
/usr/lib/python2.7/site-packages/nova/api/ec2/../../CA/genrootca.sh from
(pid=1560) runthis  /usr/lib/python2.7/site-packages/nova/utils.py:275
2011-10-14 21:30:55,188 DEBUG nova.utils [-] Running cmd (subprocess):
sh /usr/lib/python2.7/site-packages/nova/api/ec2/../../CA/genrootca.sh
from (pid=1560) execute /usr/lib/python2.7/site-packages/nova/utils.py:165

I didn't find any solution with segfault at 4 ip 0025a950 sp bfef9108
error 4 in libc-2.14.90.so[115000+1a7000]

Thank you
------------
Kind regards,
Sergey







Reply | Threaded
Open this post in threaded view
|

Problem running Openstack on fedora 16 (nova segfault)

Mark McLoughlin
Hi Sergey,

On Fri, 2011-10-14 at 21:57 +0300, Sergey Kulanov wrote:

>    Installing :
> openstack-swift-object-1.4.0-2.fc16.noarch                                                                          
> 1/5
> Non-fatal POSTIN scriptlet failure in rpm package
> openstack-swift-object-1.4.0-2.fc16.noarch
> error reading information on service swift-object: No such file or directory
> warning: %post(openstack-swift-object-1.4.0-2.fc16.noarch) scriptlet
> failed, exit status 1

It looks like this problem was reported sometime ago and a patch is
waiting to be applied:

  https://bugzilla.redhat.com/685155

Silas, David - can one of you take care of this or should I?

Cheers,
Mark.


Reply | Threaded
Open this post in threaded view
|

Problem running Openstack on fedora 16 (nova segfault)

Mark McLoughlin
In reply to this post by Sergey Kulanov
On Fri, 2011-10-14 at 21:57 +0300, Sergey Kulanov wrote:

> ---------------------------NOVA START -----------------------
> service openstack-nova-api start
>
> /var/log/messages
> Oct 14 21:30:55 server1 kernel: [ 1186.188224] nova-api[1560]: segfault
> at 4 ip 0025a950 sp bfef9108 error 4 in libc-2.14.90.so[115000+1a7000]
> Oct 14 21:30:55 server1 systemd[1]: openstack-nova-api.service: main
> process exited, code=killed, status=11
> Oct 14 21:30:55 server1 systemd[1]: Unit openstack-nova-api.service
> entered failed state.

So, there are some known problems with 2.14.90-11

  https://admin.fedoraproject.org/updates/FEDORA-2011-14175

Could you try:

  $> yum downgrade glibc?

Hopefully that will get you 2.14.90-10

Thanks,
Mark.


Reply | Threaded
Open this post in threaded view
|

Problem running Openstack on fedora 16 (nova segfault)

Sergey Kulanov
17.10.2011 14:02, Mark McLoughlin ?????:

> On Fri, 2011-10-14 at 21:57 +0300, Sergey Kulanov wrote:
>
>> ---------------------------NOVA START -----------------------
>> service openstack-nova-api start
>>
>> /var/log/messages
>> Oct 14 21:30:55 server1 kernel: [ 1186.188224] nova-api[1560]: segfault
>> at 4 ip 0025a950 sp bfef9108 error 4 in libc-2.14.90.so[115000+1a7000]
>> Oct 14 21:30:55 server1 systemd[1]: openstack-nova-api.service: main
>> process exited, code=killed, status=11
>> Oct 14 21:30:55 server1 systemd[1]: Unit openstack-nova-api.service
>> entered failed state.
> So, there are some known problems with 2.14.90-11
>
>    https://admin.fedoraproject.org/updates/FEDORA-2011-14175
>
> Could you try:
>
>    $>  yum downgrade glibc?
>
> Hopefully that will get you 2.14.90-10
>
> Thanks,
> Mark.
>
>
Hi,
Thanks for the replay
Actually I tried different glibc versions end even installing openstack
on fedora 15, I had the same problem.
     $> yum downgrade glibc glibc-common
     $> service openstack-glance-api start
     $> service openstack-glance-registry start

works fine, everything starts ok but with some warnings:

Oct 17 18:58:35 server1 yum[2795]: Installed: glibc-2.14.90-10.i686
Oct 17 18:58:45 server1 yum[2795]: Installed: glibc-common-2.14.90-10.i686
Oct 17 18:59:05 server1 glance-api[2828]: Traceback (most recent call last):
Oct 17 18:59:05 server1 glance-api[2828]: File
"/usr/lib/python2.7/site-packages/eventlet/hubs/hub.py", line 336, in
fire_timers
Oct 17 18:59:05 server1 glance-api[2828]: timer()
Oct 17 18:59:05 server1 glance-api[2828]: File
"/usr/lib/python2.7/site-packages/eventlet/hubs/timer.py", line 56, in
__call__
Oct 17 18:59:05 server1 glance-api[2828]: cb(*args, **kw)
Oct 17 18:59:05 server1 glance-api[2828]: SystemError: error return
without exception set
Oct 17 19:01:01 server1 systemd-logind[669]: New session 5 of user root.
Oct 17 19:01:01 server1 systemd-logind[669]: Removed session 5.
Oct 17 19:02:57 server1 glance-registry[2888]: Traceback (most recent
call last):
Oct 17 19:02:57 server1 glance-registry[2888]: File
"/usr/lib/python2.7/site-packages/eventlet/hubs/hub.py", line 336, in
fire_timers
Oct 17 19:02:57 server1 glance-registry[2888]: timer()
Oct 17 19:02:57 server1 glance-registry[2888]: File
"/usr/lib/python2.7/site-packages/eventlet/hubs/timer.py", line 56, in
__call__
Oct 17 19:02:57 server1 glance-registry[2888]: cb(*args, **kw)
Oct 17 19:02:57 server1 glance-registry[2888]: SystemError: error return
without exception set

Now try to start nova:
     $> [root at server1 ~]# service openstack-nova-api start
         Redirecting to /bin/systemctl  start openstack-nova-api.service

Oct 17 19:05:10 server1 kernel: [ 8053.797011] nova-api[2919]: segfault
at 4 ip 003eefc0 sp bff3c9b8 error 4 in libc-2.14.90.so[2aa000+1a6000]
Oct 17 19:05:10 server1 systemd[1]: openstack-nova-api.service: main
process exited, code=killed, status=11
Oct 17 19:05:10 server1 systemd[1]: Unit openstack-nova-api.service
entered failed state.

     $> [root at server1 ~]# service openstack-nova-volume status
Redirecting to /bin/systemctl  status openstack-nova-volume.service
openstack-nova-volume.service - OpenStack Nova Volume Server
           Loaded: loaded
(/lib/systemd/system/openstack-nova-volume.service; disabled)
           Active: active (running) since Mon, 17 Oct 2011 19:08:40
+0300; 15s ago
         Main PID: 3052 (nova-volume)
           CGroup: name=systemd:/system/openstack-nova-volume.service
                   ? 3052 /usr/bin/python /usr/bin/nova-volume
--flagfile /etc/nova/nova.conf --logfile /var/log/nova/volume.log

Only nova-volume starts, the rest services have segfault:
Oct 17 19:07:55 server1 kernel: [ 8218.917047] nova-compute[2978]:
segfault at bf856000 ip 00255d19 sp bf8538d8 error 6 in
libc-2.14.90.so[110000+1a6000]
Oct 17 19:07:55 server1 systemd[1]: openstack-nova-compute.service: main
process exited, code=killed, status=11
Oct 17 19:07:55 server1 systemd[1]: Unit openstack-nova-compute.service
entered failed state.
Oct 17 19:08:25 server1 kernel: [ 8248.830936] nova-network[3036]:
segfault at bfbaf000 ip 00f3ad3b sp bfbabae8 error 6 in
libc-2.14.90.so[df5000+1a6000]
Oct 17 19:08:25 server1 systemd[1]: openstack-nova-network.service: main
process exited, code=killed, status=11
Oct 17 19:08:25 server1 systemd[1]: Unit openstack-nova-network.service
entered failed state.
Oct 17 19:08:40 server1 nova-volume[3052]: Traceback (most recent call
last):
Oct 17 19:08:40 server1 nova-volume[3052]: File
"/usr/lib/python2.7/site-packages/eventlet/hubs/hub.py", line 336, in
fire_timers
Oct 17 19:08:40 server1 nova-volume[3052]: timer()
Oct 17 19:08:40 server1 nova-volume[3052]: File
"/usr/lib/python2.7/site-packages/eventlet/hubs/timer.py", line 56, in
__call__
Oct 17 19:08:40 server1 nova-volume[3052]: cb(*args, **kw)
Oct 17 19:08:40 server1 nova-volume[3052]: SystemError: error return
without exception set
Oct 17 19:10:32 server1 kernel: [ 8375.977931] nova-scheduler[3091]:
segfault at bfdf0000 ip 00b3bdcc sp bfdecc50 error 6 in
libc-2.14.90.so[9f6000+1a6000]
Oct 17 19:10:32 server1 systemd[1]: openstack-nova-scheduler.service:
main process exited, code=killed, status=11
Oct 17 19:10:32 server1 systemd[1]: Unit
openstack-nova-scheduler.service entered failed state.

By the way, the same happens with  glibc-2.14.90-12.i686

Thanks,
Sergey