Programmatic Access to Linux VMs

If you want to use the Connect to terminal, Automated Activities, or Life Cycle Actions features in Linux virtual machines on Hyper-V or ESX, you need to follow some steps to enable these capabilities. This document will guide you through the configuration process and explain the benefits and best practices of using these features.

Best Practices for Programmatic Access to Linux VMs

It is recommend to complete this configuration on all Linux VMs in the Lab Profile to future proof the VM images for changes during the Lab lifecycle.
When configuring and using Connect to Terminal, it is recommend that Linux distributions without a Graphical User Interface (GUI) are used. This is because when the Connect to Terminal option is enabled the ability to access the VM via the GUI will be unavailable, but the VM performance and update requirements will still be impacted. For example the VM boot time may be longer, updates may take more time, or the VM may take longer to execute tasks.

Usage Scenarios

Completing the configuration described in this document will enable the following functions within the lab environment:

Automatic logon and terminal access to a Linux VM.
Ability to run Skillable automations (Life Cycle Actions and Automated Activities) within Linux VMs.

Requirements

There are a three virtual machine (VM) requirements to enable terminal access:

Hypervisor integration tools must be installed.
Additional network Adpater configuration.
An installed, configured and running SSH server.

Hypervisor Integration Tools

To check or install the Hypervisor integration tools follow the relevent section for the Hypervisor and Linux type.

VMs hosted on Hyper-V

- Refer to the Microsoft Linux support information to see if a distribution is supported by Microsoft on Hyper-V and for instructions on installing any additional integration components.
- If a Linux operating system is not listed on the Microsoft website it is still likely to be able to work as the Hyper-V Integration Services are open source. The Hyper-V Intergration Services are normally installed by default to validate after installing the Linux Operating in a Skillable Studio Hyper-V VM complete the following steps to confirm the Integration Services are installed, and install them if not installed.

1. From a terminal window run the command
lsmod | grep hv_
1. Ensure the returned list includes hv_vmbus and hv_netsvc to confirm the Integration Services are installed.
1. If the Integration Services are not installed install them using the following commands to install them:
- On Debian/Ubuntu builds this can be installed with
sudo apt-get install hyperv-daemons
- On RHEL or CENTOS
sudo yum install hyperv-daemons -y

VMs hosted on ESX

- Linux VMs must support the open-vm-tools package. On most builds, this is installed by default. It is recommended to verify this is installed using the following commands:
- On Debian/Ubuntu builds this can be checked with the command sudo apt list --installed open-vm-tools.
- On RHEL or CENTOS sudo yum list open-vm-tools.

- If the open-vmtools package is not installed use one of the following commands to install them:
- On Debian/Ubuntu builds this can be installed with sudo apt-get install open-vm-tools.
- On RHEL or CENTOS sudo yum install open-vm-tools -y.

Configure networking

For the Connect to terminal connection or the Automated Activity service to work an extra network adapter needs to be configured. This network adapter is NOT setup on the Virtual Machine Profile (VM) within Skillable Studio, only within the VM itself.

Generally, Linux VMs need to be set to automatically bring online a new network adapter at boot and pull an address via DHCP. Most Linux distributions name adapters ethX where X is the number of adapters starting from 0. This network adapter will be given an IP Addresses in the 172.20.0.0/16 address range. Therefore, this address range cannot be used by any other network adapter in the VM.

Debian/Ubuntu

Depending on your distribution, you will need to use one of two methods to add a new network adapter:

Using /etc/network/interfaces. This is based on the ifupdown package, which has been superseded by Netplan.
Using Netplan YAML configuration files. The Netplan YAML file contains a description of the network interfaces and their respective configurations.

ifdown package configuration instructions

Add a network interface to the /etc/network/interfaces and add an entry for ethX where X is equal to the number of adapters your VM has. It should be set to automatically start at boot and pull DHCP information.
- Sample Configuration:
```
auto eth1
allow-hotplug eth1
iface eth1 inet dhcp
```

Netplan package configuration instructions

To add a network interface:
- Go to /etc/netplan, and, using an editor of your choice, open the *-config.yaml (the name will vary) file used to create the network configuration.
- Under the ethernets node, add an adapter and configure it to use DHCP, as shown in the sample below:
  - Sample YAML File, showing a static IP and DHCP configuration:
```
networks
  ethernets:
    ens32:
      addresses:
      - 192.168.1.1.21/24
      gateway4: 192.168.1.1
      nameservers:
      - 192.168.1.2
      search:
      - hexelo.com
    ens33:
      dhcp4: true
  version 2
```
Using YAML files
Please note that YAML files are sensitive to whitespace and indentation. Do not use the tab key to indent items: always use the space bar.
- After making the changes to the YAML file, run the command sudo netplan apply.

RedHat/CentOS

Create a configuration file at /etc/sysconfig/network-scripts/ifcfg-ethX, where X is equal to the number of adapters your VM has. It should be set to automatically start at boot and pull an address via DHCP.
- Sample Configuration:
```
TYPE=Ethernet
BOOTPROTO=dhcp
NAME=eth1
ONBOOT=yes
```

Configure a running SSH server

SSH configuration may need to be modified to enable password authentication or root login.

Open /etc/ssh/sshd_config and uncomment the line PasswordAuthentication yes.
If you need root login as well, uncomment PermitRootLogin yes, or add the line if you can’t find it.
Uncomment the line UseDNS yes and change it to UseDNS no, or add the line if you can't find it. This disables a reverse DNS lookup on terminal connect that can cause a 10 second delay.
Disable the networkmanager-wait or the networkd-wait service if applicable. This service can result in a 20 second boot delay when using NICs set to DHCP. On most versions of Linux this can be done via the following command:
sudo systemctl disable NetworkManager-wait-online.service
or
sudo systemctl mask systemd-networkd-wait-online.service
Having unpredictable results after disabling network manager?
On recent versions of Ubuntu, you may get unpredicatable results by disabling the network manager or the networkd service (systemd-networkd-wait-online.service).
As an alternative, you can mask the service by using the command
sudo systemctl mask systemd-networkd-wait-online.service
or
sudo systemctl mask NetworkManager-wait-online.service
Depending on which is applicable. The systemctl mask command is very powerful and should be used carefully. However, before disabling or masking these services, you shold verify whether, in fact, you need to make this configuration change. When booting the VM, watch to see whether the boot processes pauses for any reason during the network configuration. If it doesn't pause, or the pause is only a few seconds, you do not need to take any action.
Make sure the SSH service auto boots. On most versions of Linux can be done by entering the following command:
sudo systemctl enable ssh.
Start the service with the following command:
sudo systemctl start ssh.
Once this is done, capture a differencing disk.

For more information about how SSH works, and instructions on setting up an SSH server, click here.

Testing

Once the configuration has been completed and the differencing disk has been captured it is advised the changes are tested. There are two ways the configuration can be tested:

Use the Connect to terminal feature
Create a simple Automated Activity

Testing with Connect to terminal

In Skillable Studio , edit the lab profile and on the Virtual Machines tab, check the Connect to terminal checkbox for the VM you performed the above steps on.
Launch a new instance of your lab profile and verify the VM connects via SSH. The VM should boot to a terminal window.
If the VM does not boot straight into the terminal window remove the Connect to terminal option on the Lab Profile and check the configuration or review the troubleshooting section.

Testing with a simple Automated Activity

Edit the Instructions from the Lab Profile to create an Automated Activity.
From the Lightning Bolt menu create a New Automated Activity.
Ensure the Automated Activity has the following settings:
- On-demand evaluation is checked
- Target is set to the configured Linux VM
- Language is set to Bash
- Click Start from Scratch
- Enter the following into the script:
  echo true
- Save and Insert the script
Close the Instruction Editor.
Launch the Lab Profile
Press the Score button
If the script completes with success then the VM is configured correctly, otherwise review the setup and the Troubleshooting section.

Troubleshooting

Ensure the VM Profile has the correct Username and Password for being able to authenticate to the VM.
If Automated Activities do not work or the terminal fails to connect check the new network adapter has been allocated an IP Address in the range of 172.20.x.y. This can be checked using the command ip a.
When running a Linux VM on Azure and attempting to execute Automated Activities if the error “Unable to retrieve SSH connection information from the virtual machine” occurs then the Azure VM service linux-azure is required. See the instructions from Microsoft selecting the correct Linux operating system, Preparing Linux for Azure.
The new network adapter will only be allocated and configured if the Connect to terminal option has been enabled in the Lab Profile or an Automated Activity has been targeted to the Linux VM.
If the VM is unable to connect to the internet after completing this configuration, you may need to disable the default route assigned to the new adapter. The SSH network has no routing capabilities.