Running CoreOS on VMware ESXi 5.1

These instructions will walk you through running CoreOS on VMware ESXi 5.1, but will likely also work on 5.5 or 6.0.

Running the VM

Choosing a Channel

CoreOS is released into alpha, beta, and stable channels. Releases to each channel serve as a release-candidate for the next channel. For example, a bug-free alpha release is promoted bit-for-bit to the beta channel.

The channel is selected based on the URLs below. Simply replace stable with alpha or beta in the URL. Select 1 of these to download the appropriate image. Read the release notes for specific features and bug fixes in each channel.

curl -LO http://stable.release.core-os.net/amd64-usr/current/coreos_production_vmware_ova.ova

curl -LO http://beta.release.core-os.net/amd64-usr/current/coreos_production_vmware_ova.ova

curl -LO http://alpha.release.core-os.net/amd64-usr/current/coreos_production_vmware_ova.ova

Deploying with VMware vSphere Client 5.1

Use the vSphere Client to deploy the VM as follows:

1. in the menu, click “File > Deploy OVF Template…”
2. in the wizard, specify the location of the OVA downloaded earlier
3. name your VM
4. choose “thin provision” for the disk format if you want the disk to grow dynamically
5. choose your network settings
6. confirm the settings then click “Finish”

NOTE: Unselect “Power on after deployment” so you have a chance to edit VM settings before powering it up for the first time.

The last step uploads the files to your ESXi datastore and registers your VM. You can now tweak the VM settings, like memory and virtual cores. These instructions were tested to deploy to an ESXi 5.1 host.

Before powering it on, you will have to create a cloud-config.

Cloud-Config

Cloud-config can be specified by attaching a config-drive with the filesystem label config-2. This is commonly done through whatever interface allows for attaching CD-ROMs or new drives.

First create a user_data file using the the cloud-config guide. The following basic file can be used to setup a CoreOS cluster consisting of 3 servers:

#cloud-config
hostname: YOUR_HOSTNAME
write_files:
    - path: /etc/systemd/network/static.network
      permissions: 0644
      content: |
        [Match]
        Name=ens192

        [Network]
        Address=192.168.0.100/24
        Gateway=192.168.0.1
        DNS=8.8.8.8
        DNS=8.8.4.4
coreos:
    etcd2:
        # generate a new token for each unique cluster from https://discovery.etcd.io/new?size=3
        discovery: https://discovery.etcd.io/<token>
        # multi-region and multi-cloud deployments need to use 192.168.0.100
        advertise-client-urls: http://192.168.0.100:2379
        initial-advertise-peer-urls: http://192.168.0.100:2380
        # listen on both the official ports and the legacy ports
        # legacy ports can be omitted if your application doesn't depend on them
        listen-client-urls: http://0.0.0.0:2379,http://0.0.0.0:4001
        listen-peer-urls: http://192.168.0.100:2380,http://192.168.0.100:7001
    fleet:
        public-ip: 192.168.0.100
        metadata: region=europe
    flannel:
        etcd_prefix: /coreos.com/network2
    locksmith:
        endpoint: 192.168.0.100:4001
    update:
        reboot-strategy: etcd-lock
        group: alpha
    units:
        - name: etcd2.service
          command: start
        - name: fleet.service
          command: start
users:
  - name: YOUR_USERNAME
    # the following password hash is created with `openssl passwd -1` and stands for `temp`
    # this allows you to login via SSH using YOUR_USERNAME and password temp !! neither are safe !!
    passwd: $1$wvBbXhLh$MOlGF9taA57AaoDzQOH/y.
    groups:
      - sudo
      - docker
manage_etc_hosts: localhost

You’ll need to change the values per server you intend to setup and create seperate cloud-config ISOs for them. I also strongly advise to change the hash of the temp password and using something else than openssl to generate the hash.

If you choose to use a password instead of an SSH key, generating a safe hash is extremely important to the security of your system. Simplified hashes like md5crypt are trivial to crack on modern GPU hardware. Here are a few ways to generate secure hashes:

generated by haroopad


)"

# Perl (change password and salt values)
perl -e 'print crypt("password","\$6\$SALT\@@bodyquot;) . "\n"'

Using a higher number of rounds will help create more secure passwords, but given enough time, password hashes can be reversed. On most RPM based distributions there is a tool called mkpasswd available in the expect package, but this does not handle “rounds” nor advanced hashing algorithms.

Finally, to create a cloud-config ISO, use the following commands using the user_data file we just created:

mkdir -p /tmp/new-drive/openstack/latest
cp user_data /tmp/new-drive/openstack/latest/user_data
mkisofs -R -V config-2 -o configdrive.iso /tmp/new-drive
rm -r /tmp/new-drive

Note that the config-drive standard was originally an OpenStack feature, which is why you’ll see strings containing openstack. This filepath needs to be retained, although CoreOS supports config-drive on all platforms.

Note: The $private_ipv4 and $public_ipv4 substitution variables referenced in other documents are not supported on VMware. You can replace all these variables by the (static) IP of the CoreOS server you’re setting up. For example:

coreos:
  etcd2:
    # generate a new token for each unique cluster from https://discovery.etcd.io/new?size=3
    discovery: https://discovery.etcd.io/<token>
    # multi-region and multi-cloud deployments need to use $public_ipv4
    advertise-client-urls: http://$public_ipv4:2379
    initial-advertise-peer-urls: http://$private_ipv4:2380
    # listen on both the official ports and the legacy ports
    # legacy ports can be omitted if your application doesn't depend on them
    listen-client-urls: http://0.0.0.0:2379,http://0.0.0.0:4001
    listen-peer-urls: http://$private_ipv4:2380,http://$private_ipv4:7001

becomes

coreos:
  etcd2:
    # generate a new token for each unique cluster from https://discovery.etcd.io/new?size=3
    discovery: https://discovery.etcd.io/<token>
    # multi-region and multi-cloud deployments need to use $public_ipv4
    advertise-client-urls: http://192.168.0.100:2379
    initial-advertise-peer-urls: http://192.168.0.100:2380
    # listen on both the official ports and the legacy ports
    # legacy ports can be omitted if your application doesn't depend on them
    listen-client-urls: http://0.0.0.0:2379,http://0.0.0.0:4001
    listen-peer-urls: http://192.168.0.100:2380,http://192.168.0.100:7001

Attach the ISO to the VM as follows:

1. edit the settings of the CoreOS VM
2. in the dialog, select “CD/DVD drive 1” in the device list
3. select “connect at power on”
4. choose “datastore ISO file” as the device type
5. browse the datastore and select your config drive ISO
6. confirm the changes and click “OK”

Logging in

You will need to know the IP in order to SSH in. Press enter a few times at the login prompt and you should see an IP address pop up:

In this case the IP is 10.0.1.81.

Now you can login using your SSH key or password set in your cloud-config.

ssh core@10.0.1.81

Alternatively, if the cloud-config fails to apply you can append coreos.autologin to the kernel parameters on boot, the console won’t prompt for a password. This is handy for debugging.

When GNU GRUB appears at boot, make sure CoreOS default is selected and press e, then add coreos.autologin after $linux_append

Before

After

When coreos.autologin is added, press CTRL+X to boot CoreOS with these parameters. Note that the next time autologin will be disabled again as these kernel parameters aren’t persistent.

You can now manually apply the cloud-config by using the following command in the console of CoreOS:

sudo /usr/bin/coreos-cloudinit --from-file /media/configdrive/openstack/latest/user_data

Using CoreOS

Now that you have a machine booted it is time to play around. Check out the CoreOS Quickstart guide or dig into more specific topics.