How to safely reboot via First Boot script
- by unixman
With the cost and performance benefits of the SPARC T4 and SPARC T5 systems undeniably validated, the banking sector is actively moving to Solaris 11. I was recently asked to help a banking customer of ours look at migrating some of their Solaris 10 logic over to Solaris 11. While we've introduced a number of holistic improvements in Solaris 11, in terms of how we ease long-term software lifecycle management, it is important to appreciate that customers may not be able to move all of their Solaris 10 scripts and procedures at once; there are years of scripts that reflect fine-tuned requirements of proprietary banking software that gets layered on top of the operating system. One of these requirements is to go through a cycle of reboots, after the system is installed, in order to ensure appropriate software dependencies and various configuration files are in-place. While Solaris 10 introduced a facility that aids here, namely SMF, many of our customers simply haven't yet taken the time to take advantage of this - proceeding with logic that, while functional, without further analysis has an appearance of not being optimal in terms of taking advantage of all the niceties bundled in Solaris 11 at no extra cost.
When looking at Solaris 11, we recognize that one of the vehicles that bridges the gap between getting the operating system image payload delivered, and the customized banking software installed, is a notion of a First Boot script. I had a working example of this at one of the Oracle OpenWorld sessions a few years ago - we've since improved our documentation and have introduced sections where this is described in better detail. If you're looking at this for the first time and you've not worked with IPS and SMF previously, you might get the sense that the tasks are daunting. There is a set of technologies involved that are jointly engineered in order to make the process reliable, predictable and extensible.
As you go down the path of writing your first boot script, you'll be faced with a need to wrap it into a SMF service and then packaged into a IPS package. The IPS package would then need to be placed onto your IPS repository, in order to subsequently be made available to all of your AI (Automated Install) clients (i.e. the systems that you're installing Solaris and your software onto).
With this blog post, I wanted to create a single place that outlines the entire process (simplistically), and provide a hint of how a good old "at" command may make the requirement of forcing an initial reboot handy. The syntax and references to commands here is based on running this on a version of Solaris 11 that has been updated since its initial release in 2011 (i.e. I am writing this on Solaris 11.1)
Assuming you've built an AI server (see this How To article for an example), you might be asking yourself: "Ok, I've got some logic that I need executed AFTER Solaris is deployed and I need my own little script that would make that happen. How do I go about hooking that script into the Solaris 11 AI framework?"
You might start here, in Chapter 13 of the "Installing Oracle Solaris 11.1 Systems" guide, which talks about "Running a Custom Script During First Boot". And as you do, you'll be confronted with command that might be unfamiliar to you if you're new to Solaris 11, like our dear new friend: svcbundle
svcbundle is an aide to creating manifests and profiles. It is awesome, but don't let its awesomeness overwhelm you. (See this How To article by my colleague Glynn Foster for a nice working example). In order to get your script's logic integrated into the Solaris 11 deployment process, you need to wrap your (shell) script into 2 manifests - a SMF service manifest and a IPS package manifest. ....and if you're new to XML, well then -- buckle up
We have some examples of small first boot scripts shown here, as templates to build upon. Necessary structure of the script, particularly in leveraging SMF interfaces, is key. I won't go into that here as that is covered nicely in the doc link above.
Let's say your script ends up looking like this (btw: if things appear to be cut-off in your browser, just select them, copy and paste into your editor and it'll be grabbed - the source gets captured eventhough the browser may not render it "correctly" - ah, computers).
#!/bin/sh
# Load SMF shell support definitions
. /lib/svc/share/smf_include.sh
# If nothing to do, exit with temporary disable
completed=`svcprop -p config/completed site/first-boot-script-svc:default`
[ "${completed}" = "true" ] && \
smf_method_exit $SMF_EXIT_TEMP_DISABLE completed "Configuration completed"
# Obtain the active BE name from beadm: The active BE on reboot has an R in
# the third column of 'beadm list' output. Its name is in column one.
bename=`beadm list -Hd|nawk -F ';' '$3 ~ /R/ {print $1}'`
beadm create ${bename}.orig
echo "Original boot environment saved as ${bename}.orig"
# ---- Place your one-time configuration tasks here ----
# For example, if you have to pull some files from your own pre-existing system:
/usr/bin/wget -P /var/tmp/ $PULL_DOWN_ADDITIONAL_SCRIPTS_FROM_A_CORPORATE_SYSTEM
/usr/bin/chmod 755 /var/tmp/$SCRIPTS_THAT_GOT_PULLED_DOWN_IN_STEP_ABOVE
# Clearly the above 2 lines represent some logic that you'd have to customize to fit your needs.
#
# Perhaps additional things you may want to do here might be of use, like
# (gasp!) configuring ssh server for root login and X11 forwarding (for testing), and the like...
#
# Oh and by the way, after we're done executing all of our proprietary scripts we need to reboot
# the system in accordance with our operational software requirements to ensure all layered bits
# get initialized properly and pull-in their own modules and components in the right sequence,
# subsequently.
# We need to set a "time bomb" reboot, that would take place upon completion of this script.
# We already know that *this* script depends on multi-user-server SMF milestone, so it should be
# safe for us to schedule a reboot for 5 minutes from now. The "at" job get scheduled in the queue
# while our little script continues thru the rest of the logic.
/usr/bin/at now + 5 minutes <<REBOOT
/usr/bin/sync
/usr/sbin/reboot
REBOOT
# ---- End of your customizations ----
# Record that this script's work is done
svccfg -s site/first-boot-script-svc:default setprop config/completed = true
svcadm refresh site/first-boot-script-svc:default
smf_method_exit $SMF_EXIT_TEMP_DISABLE method_completed "Configuration completed"
...and you're happy with it and are ready to move on. Where do you go and what do you do?
The next step is creating the IPS package for your script. Since running the logic of your script constitutes a service, you need to create a service manifest. This is described here, in the middle of Chapter 13 of "Creating an IPS package for the script and service".
Assuming the name of your shell script is first-boot-script.sh, you could end up doing the following:
$ cd some_working_directory_for_this_project$ mkdir -p proto/lib/svc/manifest/site$ mkdir -p proto/opt/site
$ cp first-boot-script.sh proto/opt/site
Then you would create the service manifest file like so:
$ svcbundle -s service-name=site/first-boot-script-svc \
-s start-method=/opt/site/first-boot-script.sh \
-s instance-property=config:completed:boolean:false -o \
first-boot-script-svc-manifest.xml
...as described here, and place it into the directory hierarchy above.
But before you place it into the directory, make sure to inspect the manifest and adjust the appropriate service dependencies. That is to say, you want to properly specify what milestone should be reached before your service runs. There's a <dependency> section that looks like this, before you modify it:
<dependency restart_on="none" type="service"
name="multi_user_dependency" grouping="require_all">
<service_fmri value="svc:/milestone/multi-user"/>
</dependency>
So if you'd like to have your service run AFTER the multi-user-server milestone has been reached (i.e. later, as multi-user-server has more dependencies then multi-user and our intent to reboot the system may have significant ramifications if done prematurely), you would modify that section to read:
<dependency restart_on="none" type="service"
name="multi_user_server_dependency" grouping="require_all">
<service_fmri value="svc:/milestone/multi-user-server"/>
</dependency>
Save the file and validate it:
$ svccfg validate first-boot-script-svc-manifest.xml
Assuming there are no errors returned, copy the file over into the directory hierarchy:
$ cp first-boot-script-svc-manifest.xml proto/lib/svc/manifest/site
Now that we've created the service manifest (.xml), create the package manifest (.p5m) file named: first-boot-script.p5m.
Populate it as follows:
set name=pkg.fmri value=first-boot-script-AT-1-DOT-0,5.11-0
set name=pkg.summary value="AI first-boot script"
set name=pkg.description value="Script that runs at first boot after AI installation"
set name=info.classification value=\
"org.opensolaris.category.2008:System/Administration and Configuration"
file lib/svc/manifest/site/first-boot-script-svc-manifest.xml \
path=lib/svc/manifest/site/first-boot-script-svc-manifest.xml owner=root \
group=sys mode=0444
dir path=opt/site owner=root group=sys mode=0755
file opt/site/first-boot-script.sh path=opt/site/first-boot-script.sh \
owner=root group=sys mode=0555
Now we are going to publish this package into a IPS repository. If you don't have one yet, don't worry. You have 2 choices: You can either publish this package into your mirror of the Oracle Solaris IPS repo or create your own customized repo. The best practice is to create your own customized repo, leaving your mirror of the Oracle Solaris IPS repo untouched. From this point, you have 2 choices as well - you can either create a repo that will be accessible by your clients via HTTP or via NFS. Since HTTP is how the default Solaris repo is accessed, we'll go with HTTP for your own IPS repo. This nice and comprehensive How To by Albert White describes how to create multiple internal IPS repos for Solaris 11. We'll zero in on the basic elements for our needs here:
We'll create the IPS repo directory structure hanging off a separate ZFS file system, and we'll tie it into an instance of pkg.depotd. We do this because we want our IPS repo to be accessible to our AI clients through HTTP, and the pkg.depotd SMF service bundled in Solaris 11 can help us do this. We proceed as follows:
# zfs create rpool/export/MyIPSrepo
# pkgrepo create /export/MyIPSrepo
# svccfg -s pkg/server add MyIPSrepo
# svccfg -s pkg/server:MyIPSrepo addpg pkg application
# svccfg -s pkg/server:MyIPSrepo setprop pkg/port=10081
# svccfg -s pkg/server:MyIPSrepo setprop pkg/inst_root=/export/MyIPSrepo
# svccfg -s pkg/server:MyIPSrepo addpg general framework
# svccfg -s pkg/server:MyIPSrepo addpropvalue general/complete astring: MyIPSrepo
# svccfg -s pkg/server:MyIPSrepo addpropvalue general/enabled boolean: true
# svccfg -s pkg/server:MyIPSrepo setprop pkg/readonly=true
# svccfg -s pkg/server:MyIPSrepo setprop pkg/proxy_base = astring: http://your_internal_websrvr/MyIPSrepo
# svccfg -s pkg/server:MyIPSrepo setprop pkg/threads = 200
# svcadm refresh application/pkg/server:MyIPSrepo
# svcadm enable application/pkg/server:MyIPSrepo
Now that the IPS repo is created, we need to publish our package into it:
# pkgsend publish -d ./proto -s /export/MyIPSrepo first-boot-script.p5m
If you find yourself making changes to your script, remember to up-rev the version in the .p5m file (which is your IPS package manifest), and re-publish the IPS package.
Next, you need to go to your AI install server (which might be the same machine) and modify the AI manifest to include a reference to your newly created package. We do that by listing an additional publisher, which would look like this (replacing the IP address and port with your own, from the "svccfg" commands up above):
<publisher name="firstboot">
<origin name="http://192.168.1.222:10081"/>
</publisher>
Further down, in the <software_data action="install"> section add:
<name>pkg:/first-boot-script</name>
Make sure to update your Automated Install service with the new AI manifest via installadm update-manifest command. Don't forget to boot your client from the network to watch the entire process unfold and your script get tested. Once the system makes the initial reboot, the first boot script will be executed and whatever logic you've specified in it should be executed, too, followed by a nice reboot. When the system comes up, your service should stay in a disabled state, as specified by the tailing lines of your SMF script - this is normal and should be left as is as it helps provide an auditing trail for you. Because the reboot is quite a significant action for the system, you may want to add additional logic to the script that actually places and then checks for presence of certain lock files in order to avoid doing a reboot unnecessarily. You may also want to, alternatively, remove the SMF service entirely - if you're unsure of the potential for someone to try and accidentally enable that service -- eventhough its role in life is to only run once upon the system's first boot.
That is how I spent a good chunk of my pre-Halloween time this week, hope yours was just as SPARCkly^H^H^H^H fun!
