I spent some time updating Drupal to the latest version that is currently available. The last install had been running on the same old version for a long time, and it was about time. Surprisingly the following update method worked very well:

Download Drupal 5.16, use the update.php script to update the blog to the 5.16 version. No errors were presented in any way shape or form which really surprised me, hats off to the Drupal developers for building a system that is stable and works correctly. After which I had some trouble, off course the old theme was now missing so logging into my Drupal was going to be an issue without an actual theme to display any of the menu's. After using some SQL-fu I regained access. Changed the theme back to the default, and things were good to go.

Next up, I grabbed Drupal 6.10. Ran its update script and everything once again went smoothly. Way smoother than any other upgrade for any of the other PHP scripts that I have had the pleasure of upgrading, then again it has been a long time since I last upgraded something as complex as Drupal. phpBB was an absolute mess to upgrade at one point in its life, have not touched it for a while so it may have improved.

Besides the newer version, there are also some modules I have now enabled. You will see that the URL's to the posts no longer contain the node/[number] format and instead are /[yyyy]/[mm]/[dd]/[post title], this should make Google and other search engines more happy than they were before, and it will make linking to a post on my website much easier. All of the old URL's still exist but will be a 301 redirect so that Google and the likes will update their search index's with the new links.

Stale content on this website, or content that was not relevant anymore has also been removed. Mostly related to things that were happening in my personal life at the time (some of the content is still up), also removing content that if it were read now, it would do more harm than good. Other content that I have removed were posts that were time/date limited and do not apply anymore to the current situation as it stands and had no real historical value.

With all of these changes in mind, I would like to apologise for those of you with feed readers that don't correctly check the date on when posts were created, you will have had all of these changes show up as yet another item in the news feed. The feed reader I personally use had no such issues since the date on the various articles had not changed, just the URL.

Recently I was working on my OpenSolaris machine (file server, ZFS rocks, will write more on that later), and I have one non-global zone, one that use for web development, which is aptly called dev-web.

So the following shows up when I run zoneadm:

xistence@Keyhole.network.lan:~# zoneadm list -iv
  ID NAME             STATUS     PATH                           BRAND    IP    
   0 global           running    /                              native   shared
   4 dev-web          running    /storage/zones/dev-web         ipkg     shared

The thing is, I had upgraded the global zone with the latest update available for the version (snv_101b):

pkg image-update -v

This had not upgraded my one none-global zone. And running pkg image-update from within the zone itself is not possible, because you can't do an upgrade on a "live" system, mainly because the image-update wants to create a new bootable environment, something that was already created when I upgraded the global zone. So what we have to do is mount the non-global zone to /mnt and tell pkg with -R where to find it and upgrade it anyway!

First we are going to halt the current zone, since I am mean and nothing important is running on the test bed system, I just used:

zoneadm -z dev-web halt

However, the better way is to off course use the shutdown command:

zlogin dev-web shutdown

and then for good measure a halt!

Next up, looking at what we are going to be mounting.

zfs list
NAME                               USED  AVAIL  REFER  MOUNTPOINT
rpool                             11.9G  42.7G    75K  /rpool
rpool/ROOT                        2.93G  42.7G    18K  legacy
rpool/ROOT/opensolaris            7.34M  42.7G  2.74G  /
rpool/ROOT/opensolaris-1          2.92G  42.7G  2.74G  /
rpool/dump                        1.87G  42.7G  1.87G  -
rpool/export                      5.27G  42.7G    19K  /export
rpool/export/home                 5.27G  42.7G    21K  /export/home
rpool/export/home/guest           5.24G  14.8G  5.24G  /export/home/guest
rpool/export/home/xistence        37.0M  42.7G  37.0M  /export/home/xistence
rpool/swap                        1.87G  43.0G  1.51G  -
storage                            493G  3.08T  35.1K  /storage
storage/media                      398G  3.08T   398G  /storage/media
storage/virtualbox                2.10G  3.08T  2.10G  /storage/virtualbox
storage/xistence                  91.7G  3.08T  91.7G  /storage/xistence
storage/zones                      957M  99.1G  30.4K  /storage/zones
storage/zones/dev-web              957M  19.1G  32.0K  /storage/zones/dev-web
storage/zones/dev-web/ROOT         957M  19.1G  28.8K  legacy
storage/zones/dev-web/ROOT/zbe    1.60M  19.1G   936M  legacy
storage/zones/dev-web/ROOT/zbe-1   955M  19.1G   935M  legacy

When pkg image-update was run on the global zone it created a new bootable environment named opensolaris-1, the cool thing is, that beadm at the same time will also create a new bootable environment for your zones. That way you can upgrade your zones afterwards, and if stuff does not work, you can revert the ENTIRE machine back to the previous state (ZFS is cool like that), thereby also making sure that your zones are reverted so that there are no incompatibilities.

So what we are looking for in this case is a zbe-1, this is the new root for the zone that we need to update, so we now need to mount it.

mount -F zfs storage/zones/dev-web/ROOT/zbe-1 /mnt

Note, that there is no / in front of storage, this is because we are specifying a pool name, since there is no "real" path that is defined as /storage/zones/dev-web/ROOT/zbe-1. Now that it is mounted, we are able to pass the -R flag to pkg, to get it to update our zone:

xistence@Keyhole.network.lan:~# pkg -R /mnt image-update -v
Creating Plan / Before evaluation:     
UNEVALUATED:
+pkg:/entire@0.5.11,5.11-0.101:20081204T010954Z

After evaluation:
pkg:/entire@0.5.11,5.11-0.101:20081119T235706Z -> pkg:/entire@0.5.11,5.11-0.101:20081204T010954Z
Actuators:

None
PHASE                                        ACTIONS
Update Phase                                     1/1 
PHASE                                          ITEMS
Reading Existing Index                           9/9 
Indexing Packages                                1/1 

---------------------------------------------------------------------------
NOTE: Please review release notes posted at:
   http://opensolaris.org/os/project/indiana/resources/relnotes/200811/x86/
---------------------------------------------------------------------------

xistence@Keyhole.network.lan:~# umount /mnt

Voilá, and the deed is done. The last command is to off course unmount the zone, that we can then issue a zoneadm boot command to start it back up:

zoneadm -z dev-web boot

And then on the zone after we log into it (over SSH in my case):

xistence@webdev.network.lan:~# pkg list -u
No installed packages have available updates

Which is exactly what we wanted!

There seems to be an issue in PHP 5.2.6 with extensions where the extensions have to be loaded in a certain order or the PHP interpreter has a tendency to crash and fail miserably. It is weird, as depending on the system configuration the order of the extensions has to be entirely different.

Certain extensions have a dependency on other extensions and their functionality so it is understandable that they are required to be loaded before the other modules can be loaded, however this has never been an issue before. I am not entirely sure how PHP does it, but I am betting it loads all the modules and then runs an init function of sort, and since all the modules are loaded at the same time, there are no issues.

However, in this case it seems that multiple functions are clashing, or something along those lines.

Running PHP in a gdb session showed the following:

Program received signal SIGSEGV, Segmentation fault.
0x00000000 in ?? ()
(gdb) bt
#0  0x00000000 in ?? ()
#1  0x29803544 in __do_global_dtors_aux () from /usr/local/lib/php/20060613/simplexml.so
#2  0x29807c04 in _fini () from /usr/local/lib/php/20060613/simplexml.so
#3  0x2824f558 in tls_dtv_generation () from /libexec/ld-elf.so.1
#4  0x28251558 in ?? () from /libexec/ld-elf.so.1
#5  0xbfbfe9c8 in ?? ()
#6  0x282327b6 in elf_hash () from /libexec/ld-elf.so.1
#7  0x282350a0 in dlclose () from /libexec/ld-elf.so.1
#8  0x081374e4 in zend_hash_apply_deleter ()
#9  0x0813757f in zend_hash_graceful_reverse_destroy ()
#10 0x0812da7c in zend_shutdown ()
#11 0x080f4f67 in php_module_shutdown ()
#12 0x081aa892 in main ()

So it seems that simplexml is the culprit here, lets remove it. Once removed from extensions.ini we re-run gdb and give it the same information we gave it before, this time however we see this:

 Cannot load module 'SPL' because required module 'simplexml' is not loaded in Unknown on line 0

My gut instinct is to now move simplexml ABOVE the line that says extension=spl.so. After we do this, PHP is able to run without any issues. Now, for the fun:

extension=spl.so
extension=xml.so
extension=ctype.so
extension=zlib.so
extension=session.so
extension=mbstring.so
extension=pdo.so
extension=pdo_sqlite.so
extension=ftp.so
extension=posix.so
extension=pcre.so
extension=json.so
extension=zip.so
extension=suhosin.so
extension=pdo_mysql.so
extension=bz2.so
extension=openssl.so
extension=gd.so
extension=pdf.so
extension=hash.so
extension=mhash.so
extension=curl.so
extension=mcrypt.so
extension=xmlwriter.so
extension=tokenizer.so
extension=filter.so
extension=simplexml.so
extension=iconv.so
extension=sqlite.so
extension=dom.so
extension=xmlreader.so
extension=gettext.so
extension=tidy.so
extension=mysqli.so
extension=mysql.so
extension=sockets.so
extension=pspell.so
extension=imap.so
extension=ncurses.so
extension=xsl.so

Is the extension list on one of the servers I administrate. Notice that spl.so is at the top of the file, and simplexml.so is somewhere in the middle. And this works perfectly, however the same file on a second server causes it to bomb spectacularly. However, I had kept a backup of the original extensions.ini on the second server, put it back, and then experimented with it, and this is what came out:

extension=simplexml.so
extension=spl.so
extension=suhosin.so
extension=json.so
extension=zip.so
extension=filter.so
extension=hash.so
extension=session.so
extension=sysvmsg.so
extension=gd.so
extension=pdf.so
extension=tidy.so
extension=magickwand.so
extension=odbc.so
extension=calendar.so
extension=sysvshm.so
extension=soap.so
extension=tokenizer.so
extension=mcrypt.so
extension=iconv.so
extension=readline.so
extension=sysvsem.so
extension=bz2.so
extension=zlib.so
extension=ftp.so
extension=pcntl.so
extension=ncurses.so
extension=posix.so
extension=gettext.so
extension=mhash.so
extension=pcre.so
extension=mbstring.so
extension=xmlwriter.so
extension=xml.so
extension=xmlrpc.so
extension=sqlite.so
extension=openssl.so
extension=ctype.so
extension=ming.so
extension=dom.so
extension=xmlreader.so
extension=curl.so
extension=mysqli.so
extension=mysql.so
extension=sockets.so
extension=imap.so
extension=xsl.so

Now when I tried this on the first system where that first configuration file comes from, it bombs spectacularly leaving a core-dump. I have NO IDEA what is going on. I just know that log messages like the following:

pid 54662 (php-cgi), uid 80: exited on signal 11 (core dumped)
pid 54655 (php-cgi), uid 80: exited on signal 11 (core dumped)
pid 54658 (php-cgi), uid 80: exited on signal 11 (core dumped)
pid 54661 (php-cgi), uid 80: exited on signal 11 (core dumped)
pid 54660 (php-cgi), uid 80: exited on signal 11 (core dumped)

are a thing of the past, until next time when I upgrade or re-compile PHP and the order is played with again.

If you have any hints, insights or knowledge of what is going on, and what magic order I have to place the extensions in, please let me know by sending me an email at xistence@gmail. This is going to be a mystery, for now, as I definitely don't have the time to figure out what is going on and why.