electronic.alchemy :: Bhyve-ready NetBSD Images
electronic.alchemy
where the past meets the future
smartos > Bhyve-ready NetBSD Images

Bhyve-ready NetBSD Images

Created by hww3. Last updated by hww3, 2 years ago. Version #3.

I've put together some bhyve images of NetBSD for use with MNX Triton Datacenter and SmartOS. They include a pre-configured cloud-init and should be fairly well integrated with Triton.

Images are available for NetBSD 9.3 and contain the following:

I'm hosting the image on my IMGAPI server. You can add it as a source to your imgadm:

root@headnode# imgadm source -a http://images.welliver.org

Here's the current list of images:

root@headnode# imgadm avail | grep netbsd
3a362e0e-afaf-11ed-97b3-cfb1bff0c1d0  netbsd-9.3-cloudimage           20230219                                    bsd      zvol          2023-02-19

These images are intended to be light weight, so only the minimum install sets (plus the compiler and headers, necessary for building software). Here's an example of how to add other installation sets:

root# cd / && curl -L http://mirror.planetunix.net/pub/NetBSD/NetBSD-9.3/amd64/binary/sets/ttxt.tar.xz | tar xzvf -

I've tested both images as a pkgsrc build platform and they work at least as well as my previous virtual environment running on VMWare.

An important note regarding security

By default, NetBSD does not assign a password to the root user. This means that unless the password is changed, a) anyone with console access can log in as root and b) anyone in the wheel group can su to root without being prompted for a password.

The images generated will automatically set a root password if it's provided in the instance metadata. This happens automatically in Triton, but it also works with SmartOS if it's provided in the instance information (see the example below for how to do this).

Also note, the metadata client is executable by anyone, so even if the password is set, it's easy to retrieve, unless it's been changed. This seems to be the case with all image types, and the only protection comes from the permission set on the metadata socket or device.

See the metadata client source for more information about what those paths are.

How the images are built

The image build script must be run from a netbsd system, in order to format the ffs disk and install the netbsd boot loader. So, start by cloning the following repository:

[external]https://git.sr.ht/~hww3/netbsd-cloud-images

Edit the script for the version of NetBSD you'd like to use, and then run build.sh. This should create a new blank UEFI-based NetBSD install with everything configured for the first run. The result will be a raw image called "final.raw".

Copy this to your smartos/triton headnode and create a blank bhyve vm:

# create a blank vm setup to start bhyve with EFI boot.
# also, note the example of how to pass ssh keys and root password
# to cloud-init if you're not running triton.
UUID=$(uuid)
export UUID
vmadmin create << EOF
{
  "brand": "bhyve",
  "alias": "install-bhyve",
  "uuid": "${UUID}",
  "vcpus": 1,
  "autoboot": false,
  "bootrom": "uefi",
  "ram": 4096,
  "disks": [
    {
      "boot": true,
      "model": "virtio",
      "size": 10240
    }
  ],
  "nics": [
    {
      "nic_tag": "hacienda",
      "ip": "dhcp",
      "primary": "true",
      "model": "virtio"
    }
  ],
  "internal_metadata": { "root_pw": "f00Bar" },
  "customer_metadata": {
    "root_authorized_keys": "ssh-rsa YOURPUBLICKEY foo@bar.org"
  }
}
EOF

# write the raw image to the zvol of our blank vm.
dd if=/path/to/final.raw of=/dev/zvol/dsk/zones/${UUID}/disk0

# if we feel confident that all will be well, we can take the zfs dump right now
# otherwise, we'll need to test first, then destroy the vm and recreate it  in order
# be able to get a zfs dump that doesn't have a bunch of data from having run the 
# initial bootup. It's not enough to re-dd the disk without first destroying the vm, as the disk will
# contain a bunch of data beyond the end of our raw disk image.
zfs set quota=100G zones/${UUID}
zfs snap zones/${UUID}/disk0@imgsnap
zfs send zones/${UUID}@imgsnap | gzip > netbsd-image-ver.zfs.gz 
vmadm start

# test vm, make sure everything works as expected including cloud-init.
# escape sequence is ~. 
vmadm console ${UUID}

# triton seems to dislike having a rogue instance running on a machine it controls, so 
# while it exists, provisioning of new vms will fail.
vmadm stop ${UUID}
vmadm destroy ${UUID}

Adapted from the following source: [external]https://github.com/virt-lightning/netbsd-cloud-images

Notes for creating the initial image:

NetBSD DVD install images are UEFI ready starting with NetBSD 9.0, so we can finally use BHYVE for the entire process. A key trick is to make sure that the CD drive is located at a high PCI slot to ensure it gets priority:

UUID=youInstanceUUID
ISO="NetBSD-9.3-amd64.iso"
cp $ISO /zones/$UUID/root/
vmadm update $UUID bhyve_extra_opts="-s 7:0,ahci-cd,/$ISO"
vmadm start $UUID
vmadm update $UUID bhyve_extra_opts=""

You can use VNC to perform the original install. Not all VNC clients work equally well. I've had good luck with TigerVNC, but not with MacOS Screen Sharing and the "official" VNC client. Chicken also worked well, but is no longer supported on recent MacOS.

Not categorized | RSS Feed | BackLinks

comments powered by Disqus