vm_tpl.cfg, vm_tpl_kvm.cfg, vm_tpl_bhyve.cfg - OmniCube VM zone template
configuration file
SYNOPSIS
/opt/omnicube/share/vm_tpl.cfg
/opt/omnicube/share/vm_tpl_kvm.cfg
/opt/omnicube/share/vm_tpl_bhyve.cfg
DESCRIPTION
These files are
zonecfg(8)
command files that describe a complete OmniCube virtual-machine zone. Two
brand-specific templates are shipped,
vm_tpl_kvm.cfg
for
brand=kvm
zones and
vm_tpl_bhyve.cfg
for
brand=bhyve
zones. In the installed tree
/opt/omnicube/share/vm_tpl.cfg
is a symbolic link to whichever of the two matches the hypervisor the node is
provisioned for, so tooling and operators can refer to a single stable path.
A template is not read by any OmniCube script at run time. It is a starting
point: the placeholder tokens described in
Placeholders
below are substituted for the values of the VM being created, and the result is
fed to
zonecfg(8)
as a command file, e.g.
pfexec zonecfg -z zvm01 -f /zones/zvm01/config
The zone that this produces is then installed and attached with
zoneadm(8).
The backing storage referenced by the
device
resource and by the
bootdisk
attribute must exist first. Its pool and the
zone/zones/zone
dataset are created by
create_zone_pool.sh(8)
(a zpool over raw LUNs, mirrored across nodes for a SYNC zone) or by
create_zone_config.sh(8)
(a zpool over one existing raw image); the boot volume or image itself is then
created inside that dataset, as a ZFS volume for
kvm
or as a flat file under the zonepath for
bhyve.
Once the zone is running,
the mutable attributes
ram,
vcpus
and
vnc
are no longer edited in the template but through
manage_zone.sh(8)
(-m,
-p,
-n),
which rewrites them with
zonecfg select attr
on the live configuration.
File syntax
The format is the
zonecfg(8)
command-file format, not a key/value configuration file:
*
One directive per line. Leading white space is not significant.
*
create,
set
and
add
introduce directives;
end
closes the resource opened by the preceding
add.
*
A line beginning with
#
is a comment and is ignored by
zonecfg(8).
Comments are not preserved in the committed zone configuration.
*
Values containing no white space may be written unquoted
(set brand=kvm);
resource property values are conventionally double quoted in these templates.
*
Order matters only in so far as a
set
inside a resource must appear between its
add
and its
end.
Placeholders
Three tokens are substituted before the file is handed to
zonecfg(8).
They are deliberately written in a form that cannot occur in a valid zone
configuration, so an unsubstituted template fails loudly rather than creating
a wrong zone.
%%VM%%
The zone (VM) name. It doubles as the name of the dedicated zpool, so it must
satisfy both the zone and the zpool naming rules, i.e. begin with an
alphanumeric character followed by alphanumerics, dot, underscore or hyphen
(the character class enforced by
validate_name
in
omnicube_utils(3)).
%%MAC%%
The MAC address of the VM's VNIC, in colon-separated lower-case hexadecimal.
Generate it with
generate_mac_address_vm(1),
which draws from the site's IEEE MA-M block.
%%VNC_PORT%%
The TCP port the kvm VNC server listens on
(kvm
template only). Present only in
vm_tpl_kvm.cfg.
DIRECTIVES
Unless stated otherwise a directive is present in, and has the same meaning
in, both templates. Each entry below opens with
Brand:,
naming the template or templates that ship the directive.
Global settings
create -b
Brand: both. Start from a
blank
configuration rather than from the default template, so nothing is inherited
from the node's
SUNWdefault
zone template. Must be the first directive in the file.
set zonepath=path
Brand: both. Root of the zone on disk. Both templates ship
/zones/%%VM%%/root
which is the
root
directory inside the dataset that
create_zone_pool.sh(8)
mounts at
/zones/zone.
No default: a zonepath is mandatory.
set brand=kvm
Brand: kvm only. Selects the illumos KVM brand. The value is read back by
activate_zone_vnc.sh(1)
and by
manage_zone.sh(8)
to decide which VNC attribute to manipulate and how to reconstruct the zone on
a relocation target, so it must match the template it appears in.
set brand=bhyve
Brand: bhyve only. Selects the bhyve brand. Same consumers as above.
set autoboot=false
Brand: both. Do not let
zoneadm(8)
boot the VM when the global zone boots. OmniCube owns VM start-up: on a SYNC
zone the
svc:/service/omnicube/zone
instance and
config/adm_status
decide, and
zone_monitor.sh(8)
boots any zone whose
config/adm_status
is
started
(falling back to the
autoboot
property when the instance does not exist). Leaving
autoboot=true
in a template makes a relocated zone come up on two nodes. Accepted values
true
and
false;
zonecfg(8)
defaults to
false,
which the templates state explicitly.
set ip-type=exclusive
Brand: both. Give the zone an exclusive-IP stack, which is required for the
net
resource form used below (a VNIC created over a global-zone data link).
Accepted values
shared
and
exclusive;
exclusive
is the only value OmniCube VMs use.
net resource
Opened with
add net
and closed with
end.
Both templates ship exactly one
net
resource; add one block per additional interface.
set physical=link
Brand: both. Name of the VNIC handed to the zone. Both templates ship
%%VM%%i0
— the site convention is the VM name followed by
in
for interface
n.
The link is created by the brand's boot method over
global-nic.
set mac-addr=address
Brand: both. MAC address of that VNIC; see
%%MAC%%
above. When omitted the brand generates a random address, which is why the
templates always set it: a VM whose MAC changes across a relocation loses its
DHCP reservation and any MAC-based filtering.
set vlan-id=id
Brand: both. 802.1Q VLAN the VNIC is tagged with. Both templates ship
666,
which is a placeholder for the site's VM VLAN and normally has to be changed.
Accepted values 1-4094; omit the directive entirely for untagged traffic.
set global-nic=link
Brand: both. Global-zone data link the VNIC is created over. Both templates
ship
aggr0,
the LACP aggregation used on OmniCube nodes. The link must exist in the global
zone; see
dladm(8).
device resource
Opened with
add device
and closed with
end.
One block per block device exported to the guest.
set match=path
Brand: both, with a brand-specific value.
vm_tpl_kvm.cfg
ships
/dev/zvol/rdsk/%%VM%%/zones/%%VM%%/disks/bdisk.lun
— the character device of the ZFS volume holding the boot disk.
vm_tpl_bhyve.cfg
ships
/raw/disk0.raw,
a path
inside
the zonepath, because the bhyve brand consumes flat image files. There is no
default; a VM with no
device
resource has no disk.
attr resources
Every remaining setting is an
attr
resource: a three-line
add
attr block
(set name,
set type,
set value)
terminated by
end.
type
is
string
for every attribute used by these brands. The
name
values below are the ones the brands themselves interpret.
bootdisk
Brand: both. Boot device the brand hands to the guest firmware.
vm_tpl_kvm.cfg
ships the ZFS volume
name
(%%VM%%/zones/%%VM%%/disks/bdisk.lun)—
the dataset name, not the
/dev/zvol
path used by the
device
resource above.
vm_tpl_bhyve.cfg
ships the image path
/raw/disk0.raw,
matching its
device
match. No default; mandatory for a bootable VM.
disk0
Brand: bhyve only. First
additional
(non-boot) disk;
vm_tpl_bhyve.cfg
ships
/raw/disk1.raw.
Further disks follow the same pattern with
disk1,
disk2
and so on. Unset by default in
vm_tpl_kvm.cfg,
where extra disks are added as further
device
resources.
ram
Brand: both. Memory assigned to the guest, as a size with a unit suffix. Both
templates ship
8G.
There is no brand default worth relying on; always set it.
manage_zone.sh(8)-m
takes a bare integer number of gigabytes and writes back the value with the
G
suffix appended.
vcpus
Brand: both, with a brand-specific syntax.
vm_tpl_kvm.cfg
ships a plain count,
8.
vm_tpl_bhyve.cfg
ships a topology,
sockets=1,cores=2,threads=1.
manage_zone.sh(8)-p
accepts either form
(N
or
N,N,N).
Default 1 vCPU if the attribute is absent.
vnc
Brand: both, with a brand-specific syntax.
vm_tpl_kvm.cfg
ships
on,
the kvm brand's boolean switch
(on/off);
the listening port is carried separately in
vncport.
vm_tpl_bhyve.cfg
ships a bhyve VNC specification,
unix=/tmp/vm.vnc,w=1920,h=1200—
a UNIX-domain socket inside the zone plus the framebuffer geometry.
activate_zone_vnc.sh(1)
rewrites this attribute for bhyve zones into
rfb=0.0.0.0:port,unix=/tmp/vm.vnc,w=1680,h=1050
when a TCP port is requested, and relays the UNIX socket with
socat(1)
otherwise. Default: no VNC console.
vncport
Brand: kvm only. TCP port of the kvm VNC server; the template ships
%%VNC_PORT%%.
activate_zone_vnc.sh(1)
sets and reads this attribute for
brand=kvm
zones. Unset by default, in which case the brand picks its own port.
bootrom
Brand: bhyve only. Firmware image the bhyve brand boots;
vm_tpl_bhyve.cfg
ships
BHYVE_RELEASE,
the release build of the shipped UEFI ROM. The debug variant is
BHYVE_DEBUG.
Unset by default in
vm_tpl_kvm.cfg,
which does not use a bootrom.
diskif
Brand: bhyve only. Emulated disk controller;
vm_tpl_bhyve.cfg
ships
nvme.
The alternative in common use is
virtio-blk.
Windows guests need a driver for whichever value is chosen, which is why it is
part of the template rather than left to the brand default.
type
Brand: bhyve only. Guest operating-system hint;
vm_tpl_bhyve.cfg
ships
windows.
manage_zone.sh(8)
reads this attribute back
(zonecfg info attr name="type")
to decide whether a shutdown must go through the Windows SAC console, i.e.
through
shutdown_windows_server(8)
and
restart_windows_server(8),
rather than through a plain
zoneadm halt.
Unset by default for non-Windows guests.
priority
Brand: both. Site-defined start-up ordering weight for the VM;
vm_tpl_kvm.cfg
ships
1
and
vm_tpl_bhyve.cfg
ships
4.
Lower values are brought up first. The attribute is informational as far as
the brands are concerned; it is consumed by the site's start-up ordering, and
is preserved by
zone_save.sh(8)
in the saved zone configuration.
EXAMPLES
Example 1: annotated bhyve template
A complete
vm_tpl_bhyve.cfg
with the placeholders left in place. Comment lines are for the reader and may
be kept in the file that is fed to
zonecfg(8).
# Blank configuration, nothing inherited from SUNWdefault.
create -b
# Zone root: the 'root' child of <VM>/zones/<VM>, mounted at /zones/<VM>.
set zonepath=/zones/%%VM%%/root
set brand=bhyve
# OmniCube, not zoneadm, decides when this VM starts.
set autoboot=false
set ip-type=exclusive
# One VNIC named <VM>i0 over the node's aggr0, tagged on the VM VLAN.
add net
set physical="%%VM%%i0"
set mac-addr="%%MAC%%"
set vlan-id="666"
set global-nic="aggr0"
end
# Boot image, relative to the zonepath.
add device
set match="/raw/disk0.raw"
end
add attr
set name="bootdisk"
set type="string"
set value="/raw/disk0.raw"
end
# Second (data) disk.
add attr
set name="disk0"
set type="string"
set value="/raw/disk1.raw"
end
# Windows guest: manage_zone.sh shuts it down through the SAC console.
add attr
set name="type"
set type="string"
set value="windows"
end
add attr
set name="ram"
set type="string"
set value="8G"
end
add attr
set name="bootrom"
set type="string"
set value="BHYVE_RELEASE"
end
add attr
set name="priority"
set type="string"
set value="4"
end
# VNC on a UNIX socket inside the zone; activate_zone_vnc.sh relays it.
add attr
set name="vnc"
set type="string"
set value="unix=/tmp/vm.vnc,w=1920,h=1200"
end
add attr
set name="diskif"
set type="string"
set value="nvme"
end
add attr
set name="vcpus"
set type="string"
set value="sockets=1,cores=2,threads=1"
end
Example 2: the kvm differences
The kvm template differs from the bhyve one in the brand, the disk paths and
the VNC attributes only:
set brand=kvm
add device
set match="/dev/zvol/rdsk/%%VM%%/zones/%%VM%%/disks/bdisk.lun"
end
add attr
set name="bootdisk"
set type="string"
set value="%%VM%%/zones/%%VM%%/disks/bdisk.lun"
end
add attr
set name="vcpus"
set type="string"
set value="8"
end
add attr
set name="vnc"
set type="string"
set value="on"
end
add attr
set name="vncport"
set type="string"
set value="%%VNC_PORT%%"
end
Example 3: instantiating a template
Create the backing storage, substitute the placeholders and configure the
zone. A 200 GB mirrored SYNC pool for VM
zvm01
with a 4 KB record size:
The final
-r
propagates the new configuration to the other cluster nodes so a relocation
does not resurrect the old sizing.
FILES
/opt/omnicube/share/vm_tpl.cfg
Symbolic link to the template matching this node's hypervisor.
/opt/omnicube/share/vm_tpl_kvm.cfg
Template for
brand=kvm
zones.
/opt/omnicube/share/vm_tpl_bhyve.cfg
Template for
brand=bhyve
zones.
/zones/zone/config
Conventional location of the instantiated, per-zone command file.
manage_zone.sh(8)
and
zone_save.sh(8)
read and write a zone configuration export at this path.
SECURITY
A template becomes a privileged zone configuration the moment it is passed to
pfexec zonecfg -f,
and
zonepath
and
match
are file-system paths the global zone acts on as root. Templates and
instantiated command files must therefore be root-owned and must not be group
or world writable: a writable template is equivalent to arbitrary root
file-system access on the node. See
omnicube(7)
for the RBAC profile model.
The
vlan-id
and
global-nic
values shipped in both templates
(666
and
aggr0)
are site examples, not defaults that work everywhere; likewise the bhyve
/raw/disk0.raw
and
/raw/disk1.raw
image names and the kvm
disks/bdisk.lun
volume name. Review every path in the template against what
create_zone_pool.sh(8)
or
create_zone_config.sh(8)
actually created before configuring the zone.
No OmniCube script substitutes the placeholders for you; the substitution is
part of the site's provisioning procedure. A template applied verbatim
produces a zone named after the literal token
%%VM%%.
Editing a template has no effect on zones that already exist. Changes to a
running VM go through
manage_zone.sh(8),
and must be followed by
manage_zone.sh -zzone-r
so that the peer nodes' copies of the zone configuration agree; otherwise a
relocation or a SYNC-zone failover silently reverts the change.