EVOLUTION-MANAGER

Edit File: image_build.rst

======================= Building Images in Koji ======================= Image Building ============== Koji is capable of building many different types of images or appliances. They broadly fall into a few categories: LiveCDs, Disk Images, and Containers. All types of images end up with a Name-Version-Release just like an RPM build. What you need to provide Koji to perform a build varies by category, and the specifics will be explained below. For additional questions and information, ask around in ``#koji`` on FreeNode IRC and sign up to the right mailing lists as the `Koji project website`_ dictates. Kickstart First =============== No matter which type of image you want to build, you need to feed Koji a `kickstart`_ file, so we touch on how to create them and how to get them to Koji first. Once you have a kickstart to try out, look over the kickstart-specific caveats for each image type in the later sections. Kickstart Reference ------------------- If you're new to automated installations, you should read the `Anaconda Kickstart Guide`_ first. Getting your Kickstart to Koji ------------------------------ For scratch builds, you can get away with just using the ``--kickstart`` parameter, which accepts a path (relative or absolute) to your kickstart file on disk. For non-scratch builds though, the kickstart file needs to live in a remote Source Control Manager (SCM) such as git or Subversion. To access that, you need to pass the ``--ksurl`` parameter, which accepts a wacky string in this form: :: git://git.fedorahosted.org/git/spin-kickstarts.git?fedora22#68c40eb7 Here we pointed to a repository hosted at ``git://git.fedorahosted.org/git/spin-kickstarts.git``. Note the ``?`` in there, this delimiter separates the host URL with a directory structure to look in for the kickstart file. In other words, if you were to clone the repository, you should expect to see these (sub)directories laid out below the root of the clone. The important thing to realize is the behavior of ``--kickstart`` changes if ``--ksurl`` is specified. Instead of indicating a path to a kickstart file, you just indicate a filename. The file itself is not specified in the fragment of the string between ``?`` and ``#`` that is passed to ``--ksurl``, Koji still uses the ``--kickstart`` option for that. After the directory structure comes a ``#``, which indicates the start of the commit ID. This is the commit Koji will reset the repository to after cloning. If you created a git repository which had just the spec file template in it, your SCM URL would have the ``?`` and ``#`` right next to each other. If we passed ``--ksurl`` the string above, and gave ``--kickstart "server-ec2.ks"``, then Koji would do the following: * Clone the repository locally * Call ``git reset --hard 68c40eb7`` * Search in the "fedora22" subdirectory of the clone for a file called ``server-ec2.ks``, and pass that to Anaconda * Perform an automated install .. _building-livecds: Building LiveCDs ================ Let's describe building LiveCDs and the mechanics behind it. Getting Started --------------- What you need before proceeding: * a name for your LiveCD, and a version number * a Koji build target * what architectures you want * a flattened kickstart file * you may also need yum repositories generated by release engineering if you require signed packages be installed into the LiveCD * the livecd permission in Koji The Koji command that creates a LiveCD is called ``spin-livecd``. It calls out to `livecd-tools`_ in a mock chroot to construct the LiveCD. To run a LiveCD build, use the spin-livecd command like so: :: $ koji spin-livecd --release 4 fedora-workstation 23 f23-image-build fedora-workstation.ks In this example a LiveCD will be created with the N-V-R of fedora-workstation-23-4. If ``--release`` was not included an incrementing value would be computed by Koji instead. spin-livecd takes a minimum of 5 arguments: Name the name of the image, without versioning information. Examples could be ``fedora`` or ``fedora-workstation``. Version an arbitrary version string. For Fedora images, this usually matches the release version, such as 21, 22, or 23. Build Target just like RPM builds Koji must be told which target to use. This controls what tag the image will be tagged into, and what packages are available to install into the image. Architecture only x86_64 or i386 is supported Kickstart File this is a recipe file that performs drives the construction of the image. Use ``--help`` to discover more options to ``spin-livecd``. You can override the Release field with ``--release``, or ``--repo`` to override what yum repo is used to provide packages to install to the image. Note that regardless of what repo you use, it must contain RPMs built by Koji, otherwise you will get an error. (unless you are building a scratch image) Mechanics --------- The way LiveCDs are created in Koji is fairly straightforward. A chroot is initialized and populated with the packages and their dependencies from the ``livecd-build`` package group. Next, the kickstart file is copied into it if it was provided from local storage. If not, it is checked out into it from an SCM. It is then modified to use the repo associated with the build tag for the target specified in the command unless the ``--repo`` option was given. Both the original and the modified kickstart files are saved as part of the output for the task for later review. A `livecd-creator`_ command is executed using the mock('--chroot', ...) method. .. note:: This process runs as root. This produces the desired image which is uploaded to /mnt/koji/images/<image>/$imageID if it is not a scratch image. .. _caveats-for-livecds: Caveats for LiveCDs ------------------- There are some known caveats with using the spin-livecd command that users should be aware of. %include macros in the kickstart ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ A word of caution about kickstart files and the ``%include`` macro. `livecd-creator`_ is smart enough to search the current directory of the submitted kickstart file if it has %include macros. If the kickstart specified to koji is from local storage, only that kickstart file will be copied into the chroot, and this creates a problem if it has ``%include`` macros, because the other kickstart files it needs will be inaccessible. This issue is not present when the kickstart file is retrieved from a remote SCM (such as the spin-kickstarts git repo), because the entire repository is checked out. Presumably it will include any other kickstart files the specified one is including in the same directory. A workaround for the issue would be to use ``ksflatten`` (from pykickstart) on kickstart files with ``%include`` macros that are going to be submitted to koji from the user's local disk. Package Groups in the Kickstart File ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Package Groups in the kickstart file cause a problem if the Koji repos do not define them, which they most likely don't since Koji's comps.xml is based on the "groups" set up from the CLI. `livecd-creator`_'s behavior is to ignore package groups that are not defined in the repo it is using, so this can be troublesome when creating the image since packages could be left out. There are a couple possible workarounds: * do not use package groups in the kickstart file and just specify a huge list of packages * use ``--repo`` and specify a repo that does have a comps.xml that defines the groups it uses Only Include RPMs Built in Koji ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ The image building tasks will fail if your image tries to include a package that was not built in your build system. This is because the package does not have any origin information stored in Koji's database. The repos defined in the kickstart will automatically be overridden with the repo of the build tag for the build target, unless you use the ``--repo`` option. Since only packages you have built (or include from an external repo) should be there, you should never have this problem unless you use ``--repo``. No Signed or Debuginfo RPMs in Koji's Build Tags ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ If you need signed RPMs or debuginfo RPMs, you will run into trouble because Koji does not keep those in its build tag repos. The only work around for this is to create a repo yourself that includes these RPMs and then use ``--repo``. This will force the image to take RPMs from that repo. Remember, the task will fail if Koji detects RPMs were installed that were not built in the build system. %post Section in Kickstart ^^^^^^^^^^^^^^^^^^^^^^^^^^ While livecd-tools does support building on SELinux disabled hosts, you can run into denials when booting if you create and use new files in the ``%post`` section of your kickstart file. If you do, you should either set the labels appropriately at the end of the ``%post`` section, or instigate an autorelabel at boot time. Troubleshooting --------------- If your build fails, you will be notified on the command line. In the output will be a URL to the Koji UI, visit that and click on the red subtask link. From that page review ``root.log`` and ``livecd.log`` for errors. Often errors are caused by packages being missing, or malformed kickstart files. The log files are at the bottom of the page. If you're stuck, contact Release Engineering. Build System Preparation ------------------------ This section assumes you have know-how required to install and configure a new instance of Koji, and that you have already done so. You can learn how to do so :doc:`here <server_howto>` if you need to. Please ensure you are using the latest version of the software and that your database schema is updated as well. You should also have some familiarity with how `livecd-creator`_ works. This section only covers preparation for LiveCD builds. Follow this procedure step by step to get things prepared they way they need to be. #. ``koji add-host-to-channel <your-host> livecd`` add a builder to the livecd channel #. ``koji grant-permission livecd <user>`` grant the permission to build an image type to a user. This step is optional since admins have all permissions. #. You will need a tag and target to build the images from. The yum repo generated for the build tag of the target is what Koji will use to populate the LiveCDs with by default. (the alternative is to use the ``--repo`` option, more on that later) #. ``koji add-group <build-tag> livecd-build`` add the livecd-build group #. ``koji add-group-pkg <build-tag> livecd-build <pkg> ...`` add packages to the livecd-build group. These package lists vary has packages and dependencies change. As of October, 2015 for Fedora 23 the needed packages for each image type are: * bash, coreutils, fedora-logos, fedora-release, livecd-tools, policycoreutils, python-dbus, sed, selinux-policy-targeted, shadow-utils, squashfs-tools, sssd-client, tar, unzip, util-linux, which, yum Building Disk Images ==================== Disk images are files that represent virtual disks. They have a partition table and filesystems on them, and are available in a variety of formats: qcow2, vmdk, ova, Hyper-V, raw, "base" container images, and more. Getting Started --------------- What you need: * a name for your LiveCD, and a version number * what architectures you want * a Koji build target * kickstart file * installation tree * you may also need yum repositories generated by Rel-Eng if you require signed packages be installed into the image The Koji command to build a disk image is called ``image-build``. The ``image-build`` command uses `ImageFactory`_ and `Oz`_ to start a VM guest and perform an automated Anaconda installation. Here is a (lengthy) example for building a disk image. :: $ koji image-build --repo 'https://alt.fedoraproject.org/pub/alt/releases/22/Cloud/$arch/os/' --kickstart fedora-server.ks --scratch --distro Fedora-22 --format qcow2 fedora-server-kvm 22 'https://alt.fedoraproject.org/pub/alt/releases/22/Cloud/$arch/os/' x86_64 This example builds a scratch qcow2 disk image using packages from an additional yum repository. Without this option the yum repo to populate the build root would be used instead. If this was the first image with the N-V of fedora-server-kvm-22, then the N-V-R would be fedora-server-kvm-22-1, because Koji uses an incrementing number for the release if you do not provide one. Like all Koji commands, use ``--help`` to see more options that are available. For Docker, Koji only supports Base Images right now using a kickstart file as described above. In the future it will support layered images, but not before some Docker requirements are met, and Koji is maintaining a Registry of its own. This scoping effort is ongoing. ``image-build`` takes a minimum of 5 positional arguments, and 2 options must be specified. They are reviewed in the list below, with the positional arguments first. Name the name of the image, without versioning information. Examples could be ``fedora-server`` or ``fedora-workstation``. Version an arbitrary version string. For Fedora images, this usually matches the release version, such as 22 or 23. Build Target just like RPM builds Koji must be told which target to use. This controls what tag the image will be tagged into, and what packages are available to install into the image. Installation Tree URL this is a URL to a location you can install an operating system from. It is the same place you would direct a PXE-booted system to go. In 99% of cases this location is provided by Release Engineering. It should have an "isolinux" subdirectory and yum metadata somewhere within. Architecture only x86_64 or i386 is supported, and you can specify both on the command line. This will cause two subtasks to be run, allowing you to build for both arches in parallel. If either fail, the whole build will fail. Kickstart File this is a recipe file that performs drives the construction of the image. Pass in the path to a kickstart file, which must be flattened. Kickstart URL in a non-scratch build, you'll need this too. For more details, see the Getting your Kickstart to Koji section. Distro a string that indicates what OS is being built. These always follow the convention of "Fedora-X", where X is the release number. Since this command can get very long, a configuration file can drive the task as well, using the ``--config`` option. It accepts a path to a configuration file written in the Python ConfigParser format (like a Windows .ini). The options are all named the same with one caveat, see below. Here's what one could look like: :: [image-build] name = fedora-server-docker version = 22 target = f22-candidate install_tree = https://alt.fedoraproject.org/pub/alt/releases/22/Cloud/$arch/os/ arches = x86_64 format = qcow2,rhevm-ova,vsphere-ova distro = Fedora-22 repo = https://alt.fedoraproject.org/pub/alt/releases/22/Cloud/$arch/os/ disk_size = 20 ksversion = DEVEL kickstart = fedora-22-server-docker.ks ksurl = git://git.fedorahosted.org/git/spin-kickstarts.git?fedora22#68c40eb7 specfile = git://git.fedorahosted.org/git/spin-kickstarts.git?spec_templates/fedora22#68c40eb7 A few notes on the syntax: * it allows for comments too, the lines start with a hash (#) * options on the command line that can be used multiple times can accept values here as comma-separated strings * options with a hyphen need to use an underscore instead. ``--disk-size`` for example would be ``disk_size`` in the config file OVA Features ------------ If you're building OVAs, either for RHEVM or vSphere, you can specify OVA options with a special section in the configuration file. It looks something like this: :: [ova-options] vsphere_product_version=22 rhevm_description=Fedora Cloud 22 vsphere_product_vendor_name=Fedora Project ovf_memory_mb=6144 rhevm_default_display_type=1 vsphere_product_name=Fedora Cloud 22 ovf_cpu_count=4 rhevm_os_descriptor=Fedora-22 or this: :: [ova-options] vsphere_ova_format = vagrant-virtualbox rhevm_ova_format = vagrant-libvirt vagrant_sync_directory = /home/vagrant/sync The second one is actually the secret sauce for generating an image for use in Vagrant. At this time, you would need rename the image file extension from .ova to .box, but otherwise this should work fine. Kickstart Preparation --------------------- Kickstarts for the image-build command have some specific requirements which are covered in this section. Required Kickstart Arguments ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Anaconda of course requires many commands to be defined in the kickstart file. If you're starting from scratch you should review the reference linked above, or use an existing kickstart file in the spin-kickstarts git repo. It is critically important that the installation be completely automated, if Anaconda has to prompt for input for any reason, the build will fail because you cannot send input to the guest. Some of the kickstart commands are optional to Anaconda, but are required in Koji for your build to succeed. Here's the list and the reasons why. zerombr You must tell Anaconda to wipe out the MBR in the virtual block device, if you don't Anaconda will ask you. clearpart --all --initlabel Anaconda has to be told to wipe out all data on the virtual block device we install on otherwise it will ask for confirmation to do so. Since it is blank anyway this is harmless. reboot When the installation completes, the guest is rebooted. `ImageFactory`_ is specifically looking for this behavior to conclude the installation completed. Anaconda's default behavior is to wait for a key press to reboot the system, but this is impossible from outside of Koji. locking the root account You have to lock the root account (rootpw --lock) or create a non-root user (user), otherwise Anaconda will prompt for one. Do not use the url command The repo commands are overridden by Koji to point to internal Koji repos, or what you specified on the command line with ``--repo``, it does not override the url command if you provided it. Anaconda has a behavior where it will prefer packages from the repositories given with the url command over those with the repo command, and this is generally not what you want. If Koji sees an RPM was installed that was not built in the system, it will fail the build. Recommended Kickstart Arguments ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Often you want a ``%post`` section in your kickstart to perform post-installation configuration steps. Review that section of the reference and note that you can specify ``--log`` and ``--interpreter``. Both of these are recommended (but not required) to assist with the development and debugging process. Here are some other recommendations: * You probably want the network to use dhcp, sshd to be started, and port 22 opened in the firewall to allow access as well. * If you're building an image that will be shipped with a product, SELinux should be enabled. * Images that will be used in cloud deployments like OpenStack or EC2 should have ``cloud-init`` in the package list. * It is discouraged to have root passwords in plaintext in your kickstart file. * If your %post section is written in bash, consider setting -x. * For images that have multiple partitions, use the ``--asprimary`` option for the part command that defines the root file system. This will ensure it is the first partition on the image, which is a requirement in some cloud environments like EC2. Troubleshooting --------------- If your disk image build fails, follow the link in the command line output that takes you to the task page in the Koji web UI. Click on the failed createImage subtask in red. On that page review the screenshot.ppm file if it was provided, or oz.log. Most failures are from Anaconda rejecting a malformed kickstart file, which will be indicated in the screenshot. Your installation must be completely automatic, there can be no interactivity at all, otherwise Anaconda will sit there indefinitely until Koji (actually ImageFactory) kills the task. It is very easy to write a kickstart file with bugs or that results in a system that does not boot. This section will present a series of questions to ask yourself and examples to help diagnose where the problem lies. Once you know that, it should be easier to understand what you can do to inspect further. There are 4 steps in the process: #. create a guest #. perform an automated installation in the guest #. boot the guest and extract the list of installed RPMs #. upload and archive the disk image of the guest Is it a problem with guest creation? ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ There have been unusual cases where libvirt, ImageFactory, or Oz was misconfigured and guests could not be started properly. A misconfiguration with Puppet or whatever Fedora Infrastructure is up to can cause this. So far the errors have been clear in the task output, look either in the results string or oz.log. The bad news is that in this case you really can only inform Rel-Eng about the issue and wait for a resolution. The good news is these cases are very rare. Did the installation fail? ^^^^^^^^^^^^^^^^^^^^^^^^^^ The Anaconda installation can fail for many reasons: missing packages, network problems, or syntax errors in %post. Tasks will also fail if Anaconda prompts for input for any reason. If Koji detects a lack of disk activity in the guest for more than 5 minutes, it will fail the build and tear down the guest. Looking in oz.log may have the answer: dracut, anaconda, and yum logs are all printed there. These sorts of failures often have a screenshot taken and saved with the task output called screenshot.ppm. Viewing this will usually tell you what Anaconda is complaining about if the installer detected an issue or prompted for input. The string in the results output that says "No disk activity in 300 seconds, failing." This almost always means Anaconda hit an issue and either gave up or waited. If Anaconda claims it is missing packages, confirm they exist in the repos you are using with ``--repo``, if you are using that option. If you are not, confirm the builds you expect are in the tag inheritance for the target you are running. This is a lot like checking whether an RPM will build against the right libraries, except we're building an image instead. If you get the rare Anaconda dialog box that says something like "An unexpected error occurred", try using the ``text`` command in kickstart, which will have Anaconda boot in text mode. Sometimes the Python traceback (or whatever the error condition is) will be printed there. I have also seen cases where text-mode yields a black screen, but booting in graphical mode (the default) does produce a useful dialog box. Issues like this stem from syntax errors in the kickstart file, or bugs in pykickstart itself. If you think it is a pykickstart bug, then someone in Rel-Eng needs to update pykickstart on the builders. Did the guest boot? ^^^^^^^^^^^^^^^^^^^ Koji waits 5 minutes for a guest to boot in this step. It unfortunately does not give a lot of insight to why a guest may not boot, so these are a tougher class of issues to work through. You can usually answer this question by looking in results string. If you see "Timed out waiting for guest to boot", then this is your problem. You can also confirm this in oz.log. For now, the best way to investigate an issue like this is to drive a guest installation locally using something like Gnome's Virtual Machine Manager (VMM). The steps to perform are: * Select a Network Install * For the Operating System Install URL use the same one you gave to Koji. It will be something like https://alt.fedoraproject.org/pub/alt/releases/22/Cloud/x86_64/os/ * Set the Kickstart URL to where your kickstart file is. You may need to make it available over http. * Bump the memory to 2048M for good measure * Launch the guest and let it complete installation * Open a VNC session and watch what happens when the guest attempts to boot. If the console is not providing enough information, we have to get more creative. Anaconda supports starting an SSH daemon while the installation is happening with the sshpw command in kickstart. Set that and comment out the reboot command. This will let the installation complete locally and wait for a keystroke to reboot the guest. At this point you should be able to ssh in and inspect the environment to figure out what is going on. You should also consider making use of the ``--log`` option to %post so that output from the script is saved somewhere. Another option would be to scp logs and other files off of the guest as part of the ``%post`` script. Other Guest Misconfigurations ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ If the guest boots but you're having problems accessing it I'd suggest following same procedure as when the guest fails to boot. This could be a result of firewall misconfigurations or SSH not being available for some reason. Usually in this case the build is succeeding in Koji, but there's something still fundamentally broken in the image. If the issue is something you can investigate while the guest is online (you can log in), then I'd suggest importing it locally using the libvirt.xml and the disk image provided in Koji's task output. You can also do investigative work in an offline mode by mounting the image locally or using something like libguestfs to poke around without starting the guest. The fast, dirty way to do it is by mounting it. This can often pollute your guest environment. Here's how to do it: * Download the image from Koji * If the image format is not raw, you have to convert it first with qemu-img. Something like: :: $ qemu-img convert -O raw <image-file> <output-file></pre> * Now mount it up using loopback devices. (as root) If your image has multiple partitions in it, you may need to pass in a different mapped loopback device like ``loop0p2``. Whichever one you think is the root partition or has the issue you're trying to fix. :: # kpartx -av <raw-image> # mount -o loop /dev/mapper/loop0p1 /mnt/my_directory Hopefully at this point you figure out the issue. To tear down the image you'll run commands as root like so: :: # umount /mnt/my_directory # dmsetup remove loop0p1 # losetup -d /dev/loop0 Again, if you used different loopback devices, substitute those in to the dmsetup and losetup commands. Build System Preparation ------------------------ Follow this guide if you're a Koji admin and would like to enable image building or want to set up some testing before enabling the integration. When moving to ImageFactory to do image builds Koji lost the ability to easily reproduce the build environment for images the way we do for RPMs using Mock. This section will document how to set up an image builder for Koji. This is a lengthy task, it will take first-timers about a week to have a useful instance, and it is painful because it requires a bare metal system be provisioned since `ImageFactory`_ provisions VMs to build the image. There is a significant performance penalty for using nested virtualization. Follow the steps below to set up your builder. .. note:: You do not have to stand up a complete Koji instance to test the way Koji builds images. However, if you want to test image builds with an accurate representation of how Koji does it, or you want to test code changes related to image builds in Koji, you should follow all the steps below. ImageFactory/Oz Preparation ^^^^^^^^^^^^^^^^^^^^^^^^^^^ #. Provision a system with at least 4G of memory with the current release of RHEL 6 or later. Sometimes builders lag behind a month or two before taking in updates, but this will still get you pretty close to where you want to be. For Fedora, use the latest release. #. Install the following packages to the builder. #. oz #. imagefactory #. imagefactory-plugins-TinMan #. imagefactory-plugins-vSphere #. imagefactory-plugins-ovfcommon #. imagefactory-plugins-docker #. imagefactory-plugins #. imagefactory-plugins-OVA #. imagefactory-plugins-RHEVM #. python-psphere => 0.5 #. VMDKStream => 0.2 #. pykickstart #. Edit ``/etc/kojid/kojid.conf``, and set an second value, eg: 7200 for ``oz_install_timeout``. It's a timeout waiting guest installing. Default value is 0, that means oz will use its default value. Since ``oz-0.16.0``, it can be configured in ``/etc/oz/oz.cfg`` as ``install`` in ``[timeouts]`` section. #. Edit ``/etc/oz/oz.cfg``, and set the memory value in the ``[libvirt]`` section to at least 2048. Set ``safe_generation`` under ``[icicle]`` to yes. #. Run: ``mkdir -p ~root/.psphere/templates``, and then copy the following code into ``~root/.psphere/config.yaml``. Do not worry about the server, username, and password credentials; they are not used anywhere. :: general: server: 10.16.120.224 username: Administrator password: whatever template_dir: ~/.psphere/templates/ logging: destination: ~/.psphere/psphere.log level: DEBUG # DEBUG, INFO, etc Start up the services and ImageFactory/Oz should be ready to go. You should read more about `how to use Oz`_ and `how to use ImageFactory`_. If you want to try calling `ImageFactory`_ as if from a Koji Builder (but not set up a whole Koji instance), you can use the code below to emulate that. If you want to test the Koji integration with a full Koji instance, proceed to the next section instead. :: #!/usr/bin/python -tt import logging import os.path import random import sys from imgfac.BuildDispatcher import BuildDispatcher from imgfac.PluginManager import PluginManager from imgfac.ReservationManager import ReservationManager plugin_mgr = PluginManager('/etc/imagefactory/plugins.d') plugin_mgr.load() from imgfac.ApplicationConfiguration import ApplicationConfiguration # logging handler = logging.StreamHandler(sys.stdout) tlog = logging.getLogger() tlog.setLevel(logging.DEBUG) tlog.addHandler(handler) # configuration ks = open('oztest.ks').read() workdir = '/tmp/koji/test' config = { #Oz specific 'oz_data_dir': os.path.join(workdir, 'oz_data'), 'oz_screenshot_dir': os.path.join(workdir, 'oz_screenshots'), #IF specific 'imgdir': os.path.join(workdir, 'scratch_images'), 'tmpdir': os.path.join(workdir, 'oz-tmp'), 'verbose' : True, 'timeout': 3600, 'output': 'log', 'raw': False, 'debug': True, 'image_manager': 'file', 'plugins': '/etc/imagefactory/plugins.d', 'tdl_require_root_pw': False, 'image_manager_args': { 'storage_path': os.path.join(workdir, 'output_image')}, } random.seed() # necessary to ensure a unique mac address rm = ReservationManager() rm._listen_port = random.randint(rm.MIN_PORT, rm.MAX_PORT) ApplicationConfiguration(configuration=config) params = {'install_script': ks} template = """<template> <name>test-appliance</name> <os> <name>Fedora</name> <version>22</version> <arch>x86_64</arch> <install type='url'> <url>https://alt.fedoraproject.org/pub/alt/releases/22/Cloud/x86_64/os/</url> </install> <icicle> <extra_command>rpm -qa --qf '%{NAME},%{VERSION},%{RELEASE},%{ARCH},%{EPOCH},%{SIZE},%{SIGMD5},%{BUILDTIME}\n'</extra_command> </icicle> </os> <description>test-appliance OS</description> <disk> <size>16G</size> </disk> </template> """ bd = BuildDispatcher() # build the image base = bd.builder_for_base_image(template, parameters=params) base.base_thread.join() tlog.removeHandler(handler) This script is run as root with no arguments. It uses ``oztest.ks`` in your local directory as the kickstart you want to try to use. The URL in the XML template is where the RPM packages will be installed from, and what the guest will be booted with. Koji Preparation ^^^^^^^^^^^^^^^^ #. :doc:`Install Koji <server_howto>` if you need it. This will easily be the most time-consuming part of the process; my first time took 3 days to get it working properly. Follow the guide closely, and go with the SSL authentication method. SSL is a lot easier to set up locally. You will need to install every Koji component (except koji-vmd) on the same system. Proceed to the next step after you've had a successful Kojira repository generated. #. At this point, you have a system that should be ready to build images. We just have to do some Koji configuration so that your instance is pulling content from Koji. Replace the base tag names with whatever fits your conventions. #. Add tags, tag inheritance, new pkg (with pkg owner), new external repo, and regen the repo :: koji add-tag fedora22 koji add-tag jay-fedora22 koji add-tag jay-fedora22-override --parent jay-1-fedora22 koji add-tag jay-fedora22-build --arches x86_64 --parent jay-fedora22-override koji add-tag jay-fedora22-candidate --parent jay-fedora22 koji add-tag-inheritance --priority 40 jay-fedora22-build fedora22 koji add-pkg --owner kojiadmin jay-fedora22 fedora-server-ec2 fedora-server-kvm koji add-external-repo -t fedora::20 fedora22 'https://alt.fedoraproject.org/pub/alt/releases/22/Cloud/$arch/os/' koji add-target jay-fedora22-candidate jay-fedora22-build koji regen-repo jay-fedora22-build #. Grab a kickstart file from an image task in Koji that relates to what you want to test. #. Finally, kick off a build! :: koji image-build fedora-server-ec2 22 --distro Fedora-22 jay-fedora22-candidate --kickstart fedora-server-starter-ec2.ks 'https://alt.fedoraproject.org/pub/alt/releases/22/Cloud/$arch/os/' Building Appliances =================== This section is here for the sake of legacy. Unless you are trying to build ARM images, you should use the image-build command described in the previous section. .. note:: The spin-appliance command, described herein, is deprecated. Getting Started --------------- Here's what you need before proceeding: * a name for your Appliance, and a version number * a Koji build target * what architectures you want * a flattened kickstart file * you may also need yum repositories generated by release engineering if you require signed packages be installed into the appliance * the appliance permission in Koji The Koji command that creates an Appliance is called ``spin-appliance``. It calls out to appliance-creator in a mock chroot to construct the Appliance. To run an Appliance build, use the spin-appliance command like so: :: $ koji spin-appliance --release 4 fedora-workstation 23 f23-build fedora-workstation.ks In this example an Appliance will be created with the N-V-R of ``fedora-workstation-23-4``. If ``--release`` was not included an incrementing value would be computed by Koji instead. spin-appliance takes a minimum of 5 arguments: Name the name of the image, without versioning information. Examples could be ``fedora`` or ``fedora-workstation``. Version an arbitrary version string. For Fedora images, this usually matches the release version, such as 21, 22, or 23. Build Target just like RPM builds Koji must be told which target to use. This controls what tag the image will be tagged into, and what packages are available to install into the image. Architecture only arm, x86_64, or i386 are supported Kickstart File this is a recipe file that performs drives the construction of the image. Use ``--help`` to discover more options to spin-appliance. You can override the Release field with ``--release``, or ``--repo`` to override what yum repo is used to provide packages to install to the image. Note that regardless of what repo you use, it must contain RPMs built by Koji, otherwise you will get an error. (unless you are building a scratch image) Mechanics --------- The way Appliances are created in Koji is the same as :ref:`building-livecds`. Caveats for Appliances ---------------------- Caveats for using ``spin-appliance`` are the same as using ``spin-livecd`` to :ref:`caveats-for-livecds`. Troubleshooting --------------- If your build fails, you will be notified on the command line. In the output will be a URL to the Koji UI, visit that and click on the red subtask link. From that page review ``root.log`` and ``appliance.log`` for errors. Often errors are caused by packages being missing, or malformed kickstart files. The log files are at the bottom of the page. If you're stuck, contact Release Engineering. Build System Preparation ------------------------ This section assumes you have know-how required to install and configure a new instance of Koji, and that you have already done so. You can learn how to do so :doc:`here <server_howto>` if you need to. Please ensure you are using the latest version of the software and that your database schema is updated as well. You should also have some familiarity with how `appliance-creator`_ works. This section only covers preparation for Appliance builds. Follow this procedure step by step to get things prepared they way they need to be. #. Add a builder to the appliance channel :: koji add-host-to-channel <your-host> appliance #. Grant the permission to build an appliance to a user. This step is optional since admins have all permissions. :: koji grant-permission appliance <user> #. You will need a tag and target to build the images from. The yum repo generated for the build tag of the target is what Koji will use to populate the Appliances with by default. (the alternative is to use the ``--repo`` option, more on that later) #. Add the appliance-build group :: koji add-group <build-tag> appliance-build`` #. Add packages to the appliance-build group. These package lists vary has packages and dependencies change. As of October, 2015 for Fedora 24 the needed packages for appliances: * appliance-tools, bash, coreutils, grub, parted, perl, policycoreutils, selinux-policy, shadow-utils, sssd-client :: koji add-group-pkg <build-tag> appliance-build <pkg> ... .. _Koji project website: https://fedorahosted.org/koji/wiki .. _kickstart: https://github.com/rhinstaller/pykickstart/blob/master/docs/kickstart-docs.rst .. _Anaconda Kickstart Guide: https://github.com/rhinstaller/pykickstart/blob/master/docs/kickstart-docs.rst .. _livecd-tools: https://github.com/rhinstaller/livecd-tools/ .. _livecd-creator: https://fedoraproject.org/wiki/FedoraLiveCD .. _ImageFactory: http://imgfac.org/ .. _Oz: https://github.com/clalancette/oz .. _how to use Oz: https://github.com/clalancette/oz/wiki .. _how to use ImageFactory: http://imgfac.org/documentation/ .. _appliance-creator: https://fedoraproject.org/wiki/Features/ApplianceTools