These are the manual tests currently performed for validating the core snap at the candidate stage:
-
User can see both eth0 and wlan0 on first boot
- Devices: pi3
- Steps:
- Flash an image in a SD card, insert it into your (powered-down) board and plug the charger
Result: System boots. - After a while, system asks the user to press Enter to start configuration.
Result: Console-conf starts. - Hit enter again to view the network page
Result: Page moves onto the networking page - Network page is displayed
Result: You can see both Ethernet and Wireless connection
- Flash an image in a SD card, insert it into your (powered-down) board and plug the charger
-
User mistypes U1 account
- Devices: dragonboard, pi2, pi3, amd64, i386
- Steps:
- Check the U1 account settings screen.
Result: There is a text field to introduce the e-mail for the U1 account. - Enter the e-mail of a non existing account or mistype an existing account with ssh key available and press Done.
Result: Console-conf returns an error about a non existing account
- Check the U1 account settings screen.
-
User cancels U1 account setup
- Devices: dragonboard, pi2, pi3, amd64, i386
- Steps:
- Check the U1 account settings screen.
Result: There is a text field to introduce the e-mail for the U1 account. - Enter the e-mail of an existing account with ssh key available and press Done
Result: User registration starts. - Press Cancel before U1 account is registered.
Result: U1 account registration is canceled.
- Check the U1 account settings screen.
-
User reboots during U1 account setup
- Devices: dragonboard, pi2, pi3, amd64, i386
- Steps:
- Check the U1 account settings screen.
Result: There is a text field to introduce the e-mail for the U1 account. - Enter the e-mail of an existing account with ssh key available and press Done
Result: User registration starts. - Press Ctrl+Alt+Sup to reboot before the U1 account is registered.
Result: Ubuntu-core restarts and console-conf starts again.
- Check the U1 account settings screen.
-
User cancels network setup
- Devices: dragonboard, pi2, pi3, amd64, i386
- Steps:
- Check network connections screen.
Result: Wi-Fi option (wlan0) is displayed; Ethernet option (eth0) is displayed; Note: listed options are different depending on the device - Select Wi-Fi device (wlan0) or Ethernet option (eth0).
Result: Network interface wlan0 / eth0 screen is displayed. - For wlan0 config you also need to press enter again in “Configure WIFI settings” highlighted by default.
Result: Network interface wlan0 WIFI configuration is displayed. - Enter the network name, password and press Done for wlan0. DHCP IPv4 is enabled by default so put both IPv4 and IPv6 to “do not use” and press Done.
Result: Wi-Fi AP is saved and console-conf returns to Network interfaces wlan0. In addition, DHCP IPv4 will automatically set as active. Network connection shows eth0 shows both IPv4 and IPv6 as “do not use” - Once a interface is configured press Cancel in network connections page and reboot.
Result: Network changes aren’t applied and Network connections screen should display interfaces as: - “No access point configured” for wlan0 - DHCP IPv4 enabled and IPv6 not configured that were the default values.
- Check network connections screen.
-
User reboots after network setup
- Devices: dragonboard, pi2, pi3, amd64, i386
- Steps:
- Check network connections screen.
Result: Wi-Fi option (wlan0) is displayed; Ethernet option (eth0) is displayed; Note: listed options are different depending on the device - Setup wlan0 or eth0 interface and press Done. Note: press “Done” means that network changes are saved in a file in /etc/netplan/ so rebooting will keep those changes.
Result: Network changes are correctly applied and Profile setup screen appears requesting an e-mail addres - Press Ctrl+Alt+Sup to reboot.
Result: Check console-conf remembers the network configuration saved before rebooting
- Check network connections screen.
-
User introduces wrong Wifi credentials
- Devices: dragonboard, pi3
- Steps
- Check network connections screen.
Result: Wi-Fi option (wlan0) is displayed. - Select Wi-Fi device (wlan0).
Result: Network interface wlan0 screen is displayed. - Select “Configure WIFI settings” to setup a new access point.
Result: Wi-Fi access point screen is displayed with the fields “Network name” and “Password” - Enter a wrong network name and password.
Result: Wifi credentials aren’t saved, an error message is displayed and user is prompted to introduce other credentials.
- Check network connections screen.
-
User sets up an U1 account without ssh key
- Devices: dragonboard, pi2, pi3, amd64, i386
- Steps
- Run this test after setting up the network in console-conf.
- Check the U1 account settings screen.
Result: There is a text field to introduce the e-mail for the U1 account. - Enter the e-mail of an existing account without ssh key available and press Done.
Result: User is not created in the system. Error is displayed and asks the user to choose another U1 account. "Creating user failed: error: while creating user: cannot create user for “U1 account”: no ssh keys found
-
No network available
- Devices: pi2
- Steps
- Use a Raspberry Pi 2 without ethernet wire connected.
- Check network setup screen.
Result: Ethernet option is available. - Select ethernet option.
Result: Ethernet settings screen is displayed. - Select DCHP IPv4.
Result: DHCP IPv4 is the active option. - Go back to the network setup screen.
Result: Network setup screen is displayed. - Press Done.
Result: Network settings cannot be applied and console-conf doesn’t present the next settings screen.
-
ssh to device without the private key
- Devices: dragonboard, pi2, pi3, amd64, i386
- Steps
- Device has been set up with proper U1 credentials and is connected to the network. Computer doesn’t have the private ssh key of the U1 account or the key is temporarily disabled (check in .ssh).
- ssh to the device (ssh @).
Result: User must not be able to login to the device because it doesn’t have the private ssh key.
-
User is prompted to start console-conf Doesn’t press enter
- Devices: dragonboard, pi2, pi3, amd64, i386
- Steps
- Flash an image in a SD card, insert it into your (powered-down) board and plug the charger. Boot the image file from kvm/qemu for amd64/i386 images.
Result: System boots. - After a while, system asks the user to press Enter to start configuration. Press any keys other than Enter
Result: Console-conf Doesn’t start Characters shouldn’t be visible.
- Flash an image in a SD card, insert it into your (powered-down) board and plug the charger. Boot the image file from kvm/qemu for amd64/i386 images.
-
User is prompted to start console-conf Press some other keys then enter
- Devices: dragonboard, pi2, pi3, amd64, i386
- Steps
- Flash an image in a SD card, insert it into your (powered-down) board and plug the charger. Boot the image file from kvm/qemu for amd64/i386 images.
Result: System boots. - After a while, system asks the user to press Enter to start configuration. Press any other key then hit enter
Result: Characters shouldn’t be visible once enter is hit it should start console-conf
- Flash an image in a SD card, insert it into your (powered-down) board and plug the charger. Boot the image file from kvm/qemu for amd64/i386 images.
-
Ubuntu Core start page press some other keys Don’t press enter
- Devices: dragonboard, pi2, pi3, amd64, i386
- Steps
- Flash an image in a SD card, insert it into your (powered-down) board and plug the charger. Boot the image file from kvm/qemu for amd64/i386 images.
Result: System boots. - After a while, system asks the user to press Enter to start configuration. Press Enter
Result: Page 2 of Console Conf is started Reads “Ubuntu Core /n Configure the network and setup an administrator account on this all-snap Ubuntu Core system” - With the option to start selected by default, Press some keys but not the Enter or space key
Result: Ensure the keypresses are not displayed and the Page doesn’t move on to the networking page
- Flash an image in a SD card, insert it into your (powered-down) board and plug the charger. Boot the image file from kvm/qemu for amd64/i386 images.
-
Ubuntu Core start page press some other keys then press enter
- Devices: dragonboard, pi2, pi3, amd64, i386
- Steps
- Flash an image in a SD card, insert it into your (powered-down) board and plug the charger. Boot the image file from kvm/qemu for amd64/i386 images.
Result: System boots. - After a while, system asks the user to press Enter to start configuration. Press Enter
Result: Page 2 of Console Conf is started Reads “Ubuntu Core /n Configure the network and setup an administrator account on this all-snap Ubuntu Core system” - With the option to start selected by default, Press some keys and then the Enter or Space Key
Result: Ensure the keypresses are not displayed and the Page moves on to the networking page once Enter is hit
- Flash an image in a SD card, insert it into your (powered-down) board and plug the charger. Boot the image file from kvm/qemu for amd64/i386 images.
-
No network Ethernet
- Devices: pi2, pi3, amd64, i386
- Steps
- In kvm run kvm -name core--testing -m 2048 -smp 2 -vga qxl -cpu host -hda /path/to/image -net nic,model=virtio -net user,hostfwd=tcp::8022-:22 -snapshot (adding the name makes it easy to track if you have more than one running)
On hardware for sdcards do:
- sudo snap install godd
- sudo godd /path/to/image /dev/mmcblk0 (where mmcblk0 is the sdcard root)
- Insert the sdcard and power up the board
On PC:
- Insert a usb pendrive to the pc with the image
- sudo snap install godd
- godd /path/to/image /dev/sdb (where sdb is the usb pendrive root)
- Insert the pendrive in the test pc and set the system to boot from usb
Result: DUT boots and gets to the screen reading: Press enter to configure
* Hit the Enter key
Result: DUT moves to: Ubuntu Core Configure the network and setup an administrator account on this all-sanp Ubuntu Core system
* Hit the enter key
Result: DUT moves to: Network Connections Configure at least the main interface this server will use to receive updates <Network interfaces>
* Move the highlight up to eth0 and tap enter
Result: DUT moves to: Network Interface eth0
* Move highlight down to Do Not Use on the IPv4 section and tap enter
Result: IPv4 displays: IPv4 is not configured but has address: XX.XX.XX.XX
* Move the highlight down to Do not use in the IPv6 section and tap enter
Result: IPv6 displays: IPv6 is not configured but has address: XXXX:XXXX:XXXX:XXXX
* Move the highlight down to Done and tap enter
Result: DUT moves to: Network connections Configure at least the main interface this server will use to receive updates.
* With the networking setup move the highlight to done and hit enter
Result: DUT moves to: Profile setup Enter an email address from your account in the store Email address:
* Add an ubuntuone user email address that has an ssh key associated with it (using shift 2 for the
@ symbol)
Result: Email is displayed
* Hit the Enter Button
Result: DUT closes console-cont and moves to: Congratulations! This device is now registered to The next step is to log into the device via ssh:
* kvm user ssh @localhost -p 8022 On real devices ssh @
Result: Ssh connects and you are in the Ubuntu Core system
* Type: sudo reboot
Result: DUT reboots the ssh connection is dropped. DUT’s display is back to: Congratulations! This device is now registered to
- No network Wi-Fi
- Devices: dragonboard
- Steps
- In kvm run kvm -name core--testing -m 2048 -smp 2 -vga qxl -cpu host -hda /path/to/image -net nic,model=virtio -net user,hostfwd=tcp::8022-:22 -snapshot (adding the name makes it easy to track if you have more than one running)
On hardware for sdcards do:
- sudo snap install godd
- sudo godd /path/to/image /dev/mmcblk0 (where mmcblk0 is the sdcard root)
- Insert the sdcard and power up the board
On PC:
- Insert a usb pendrive to the pc with the image
- sudo snap install godd
- godd /path/to/image /dev/sdb (where sdb is the usb pendrive root)
- Insert the pendrive in the test pc and set the system to boot from usb
Result: DUT boots and gets to the screen reading: Press enter to configure
* Hit the Enter key
Result: DUT moves to: Ubuntu Core Configure the network and setup an administrator account on this all-sanp Ubuntu Core system
* Hit the enter key
Result: DUT moves to: Network Connections Configure at least the main interface this server will use to receive updates <Network interfaces>
* Move the highlight up to wlan0 and tap enter
Result: DUT moves to: Network Interface wlan0
* Move highlight down to Do Not Use on the IPv4 section and tap enter
Result: IPv4 displays: IPv4 is not configured
* Move the highlight down to Do not use in the IPv6 section and tap enter
Result: IPv6 displays: IPv6 is not configured
* Move the highlight down to Done and tap enter
Result: DUT moves to: Network connections Configure at least the main interface this server will use to receive updates.
* With the networking setup move the highlight to done and hit enter
Result: DUT displays an error in red: "Network configuration timed-out; please verify your settings," This is working as design, user profile page should not be displayed until you've got internet access.
- wlan static IPv4 enabled and IPv6 disabled
- Devices: dragonboard
- Steps
- In kvm run kvm -name core--testing -m 2048 -smp 2 -vga qxl -cpu host -hda /path/to/image -net nic,model=virtio -net user,hostfwd=tcp::8022-:22 -snapshot (adding the name makes it easy to track if you have more than one running)
On hardware for sdcards do:
- sudo snap install godd
- sudo godd /path/to/image /dev/mmcblk0 (where mmcblk0 is the sdcard root)
- Insert the sdcard and power up the board
On PC:
- Insert a usb pendrive to the pc with the image
- sudo snap install godd
- godd /path/to/image /dev/sdb (where sdb is the usb pendrive root)
- Insert the pendrive in the test pc and set the system to boot from usb
Result: DUT boots and gets to the screen reading: Press enter to configure
* Hit the Enter key
Result: DUT moves to: Ubuntu Core Configure the network and setup an administrator account on this all-sanp Ubuntu Core system
* Hit the enter key
Result: DUT moves to: Network Connections Configure at least the main interface this server will use to receive updates <Network interfaces>
* Move the highlight up to the wlan0 and tap enter
Result: DUT moves to: Network Interface wlan0
* Highlight Configure WIFI setting and tap enter
Result: DUT moves to: Network interface wlan0 manual IPv4 configuration
* Move the highlight either to network name or choose a visible network and add your network
Result: Network name is displayed
* If required highlight the password, add the password and move to done and hit enter
Result: Password is added and the DUT moves to: Network Interface wlan0
* Move the highlight to Use a static IPv4 configuration and hit enter
Result: DUT moves to: Network interface wlan0 manual IPv4 configuration
* Move the highlight among the fields and add the addresses,then move to done and hit enter. For hardware it is specific to your network check the address on you laptop. Once all the fields are filled, static IPv4 on a regular private wlan should look like:
Subnet: 192.168.X.0/24
Address: 192.168.X.XX
Gateway: 192.168.X.1
Name server: XX.XX.XX.XXX
Search domains: <leave blank>
Results: DUT moves to: Network interface wlan0
* Move the highlight to do not use in IPv6 config and hit enter, then move to done and hit enter
Result: DUT moves to: Network connection
* With the networking setup move the highlight to done and hit enter
Result: Eventually DUT moves to: Profile setup Enter an email address from your account in the store Email Address:
* Add an ubuntuone user email address that has an ssh key associated with it (using shift 2 for @ symbol)
Result: email is displayed
* With the Profile setup complete move the highlight to done and hit enter
Result: DUT moves to: Configuration Complete <User info>
* Hit the Enter Button
Result: DUT closes console-cont and moves to: Congratulations! This device is now registered to <user> The next step is to log into the device via ssh: Make sure the IP address displayed is the one added in step 9
* kvm user ssh <sso user>@localhost -p 8022<br/>On real devices ssh <sso user>@<ip address listed>
Result: Ssh connects and you are in the Ubuntu Core system
* Type: sudo reboot
Result: DUT reboots the ssh connection is dropped. DUT's display is back to: Congratulations! This device is now registered to <user> The next step is to log into the device via ssh:
* kvm user ssh <sso user>@localhost -p 8022<br/>On real devices ssh <sso user>@<ip address listed>
Result: Ssh connects and you are in the Ubuntu Core system
Run:
snap list
snap version
snap find hello | Snap list displays something like:
snap list
Name Version Rev Developer Notes
core 16.04.1 717 canonical -
dragonboard 16.04-0.18 24 canonical-
dragonboard-kernel 4.4.0-1035-1 18
canonical -
Snap version displays something like:
snap 2.20
snapd 2.20
series 16
Snap find hello display this amongst others:
snap find hello
hello 2.10 canonical - GNU Hello, the "hello world" snap
This confirms you are online and the system can connect to the store.
- Eth0 static manual IPv4 configuration
- Devices: pi2, pi3, amd64, i386
- Steps * In kvm run kvm -name core--testing -m 2048 -smp 2 -vga qxl -cpu host -hda /path/to/image -net nic,model=virtio -net user,hostfwd=tcp::8022-:22 -snapshot (adding the name makes it easy to track if you have more than one running)
On hardware for sdcards do:
- sudo snap install godd
- sudo godd /path/to/image /dev/mmcblk0 (where mmcblk0 is the sdcard root)
- Insert the sdcard and power up the board
On PC:
- Insert a usb pendrive to the pc with the image
- sudo snap install godd
- godd /path/to/image /dev/sdb (where sdb is the usb pendrive root)
- Insert the pendrive in the test pc and set the system to boot from usb
Result: DUT boots and gets to the screen reading: Press enter to configure
* Hit the Enter key
Result: DUT moves to: Ubuntu Core Configure the network and setup an administrator account on this all-sanp Ubuntu Core system
* Hit the enter key
Result: DUT moves to: Network Connections Configure at least the main interface this server will use to receive updates <Network interfaces>
* Move the highlight up to eth0 and hit enter
Result: DUT move to: Network interface eth0
* Move the highlight to Use a static IPv4 configuration and hit enter
Result: DUT moves to: Network interface eth0 manual IPv4
* Add the details for the device move the highlight to Done and hit enter
For hardware it is specific to your network check the address on you laptop<br/>
For KVM suggested settings are as follows:<br/>
Subnet: 10.0.2.0/24<br/>
Address: 10.0.2.15<br/>
Gateway: 10.0.2.2<br/>
Name server: 10.0.2.2,8.8.8.8<br/>
Search domains: <leave blank>
Result: DUT moves to: Network interface eth0
* Move the highlight to Done and hit enter
Result: DUT moves to: Network connection
* With the networking setup move the highlight to done and hit enter
Result: DUT moves to: Profile setup. Enter an email address from your account in the store Email address:
* Add an ubuntuone user email address that has an ssh key associated with it (using shift 2 for the @ symbol)
Result: Email is displayed
* Hit the Enter Button
Result: DUT closes console-cont and moves to: Congratulations! This device is now registered to <user> The next step is to log into the device via ssh:
* kvm user ssh <sso user>@localhost -p 8022 On real devices ssh <sso user>@<ip address listed>
Result: Ssh connects and you are in the Ubuntu Core system
* Type: sudo reboot
Result: DUT reboots the ssh connection is dropped. DUT's display is back to: Congratulations! This device is now registered to <user> The next step is to log into the device via ssh:
* kvm user ssh <sso user>@localhost -p 8022<br/>On real devices ssh <sso user>@<ip address listed>
Result: Ssh connects and you are in the Ubuntu Core system
Run:
snap list
snap find hello | Snap list displays something like:
snap list
Name Version Rev Developer Notes
core 16.04.1 1714 canonical -
pc 16.04-0.18 24 canonical-
pc-kernel 4.4.0-1035-1 18
canonical -
Snap version displays something like:
snap 2.20
snapd 2.20
series 16
Snap find hello display this amongst others:
snap find hello
hello 2.10 canonical - GNU Hello, the "hello world" snap
This confirms you are online and the system can connect to the store.
- Mistype Wlan SSID manually
- Devices: dragonboard
- Steps * In kvm run kvm -name core--testing -m 2048 -smp 2 -vga qxl -cpu host -hda /path/to/image -net nic,model=virtio -net user,hostfwd=tcp::8022-:22 -snapshot (adding the name makes it easy to track if you have more than one running)
On hardware for sdcards do:
- sudo snap install godd
- sudo godd /path/to/image /dev/mmcblk0 (where mmcblk0 is the sdcard root)
- Insert the sdcard and power up the board
On PC:
- Insert a usb pendrive to the pc with the image
- sudo snap install godd
- godd /path/to/image /dev/sdb (where sdb is the usb pendrive root)
- Insert the pendrive in the test pc and set the system to boot from usb
Result: DUT boots and gets to the screen reading: Press enter to configure
* Hit the Enter key
Result: DUT moves to: Ubuntu Core Configure the network and setup an administrator account on this all-sanp Ubuntu Core system
* Hit the enter key
Result: DUT moves to: Network Connections Configure at least the main interface this server will use to receive updates <Network interfaces>
* Move the highlight up to the wlan0 and tap enter
Result: DUT moves to Network interface wlan0
* Highlight Configure WIFI setting and tap enter
Result: DUT moves to: Network interface wlan0 manual IPv4 configuration
* Move the highlight to network name and mistype the name of your network
Result: Network name is displayed
* If required highlight the password, add the password and move to done and hit enter
Result: Password is added and the DUT moves to: Network Interface wlan0
* Move the highlight to done and hit enter
Result: DUT moves to: Network connections
* With the network setup move the highlight to done and hit enter
Result: DUT displays: "Network configuration timed out; please verify your settings." This message proves that our wlan0 config is wrong and there is no internet access. DUT shouldn't move to profile setup page in any case.
- Mistyped password for manually added SSID
- Devices: dragonboard
- Steps * In kvm run kvm -name core--testing -m 2048 -smp 2 -vga qxl -cpu host -hda /path/to/image -net nic,model=virtio -net user,hostfwd=tcp::8022-:22 -snapshot (adding the name makes it easy to track if you have more than one running)
On hardware for sdcards do:
- sudo snap install godd
- sudo godd /path/to/image /dev/mmcblk0 (where mmcblk0 is the sdcard root)
- Insert the sdcard and power up the board
On PC:
- Insert a usb pendrive to the pc with the image
- sudo snap install godd
- godd /path/to/image /dev/sdb (where sdb is the usb pendrive root)
- Insert the pendrive in the test pc and set the system to boot from usb
Result: DUT boots and gets to the screen reading: Press enter to configure
* Hit the Enter key
Result: DUT moves to: Ubuntu Core Configure the network and setup an administrator account on this all-sanp Ubuntu Core system
* Hit the enter key
Result: DUT moves to: Network Connections Configure at least the main interface this server will use to receive updates <Network interfaces>
* Move the highlight up to the wlan0 and tap enter
Result: DUT moves to Network interface wlan0
* Highlight Configure WIFI setting and tap enter
Result: DUT moves to: Network interface wlan0 manual IPv4 configuration
* Move the highlight to network name and mistype the name of your network
Result: Network name is displayed
* Move the highlight to the password field, mistype it and move to done and hit enter
Result: Password is added and the DUT moves to: Network Interface wlan0
* Move the highlight to done and hit enter
Result: DUT moves to: Network connections
* With the network setup move the highlight to done and hit enter
Result: DUT displays: "Network configuration timed out; please verify your settings." This message proves that our wlan0 config is wrong and there is no internet access. DUT shouldn't move to profile setup page in any case.
- Mistyped wlan SSID selected from APs name list
- Devices: dragonboard
- Steps * In kvm run kvm -name core--testing -m 2048 -smp 2 -vga qxl -cpu host -hda /path/to/image -net nic,model=virtio -net user,hostfwd=tcp::8022-:22 -snapshot (adding the name makes it easy to track if you have more than one running)
On hardware for sdcards do:
- sudo snap install godd
- sudo godd /path/to/image /dev/mmcblk0 (where mmcblk0 is the sdcard root)
- Insert the sdcard and power up the board
On PC:
- Insert a usb pendrive to the pc with the image
- sudo snap install godd
- godd /path/to/image /dev/sdb (where sdb is the usb pendrive root)
- Insert the pendrive in the test pc and set the system to boot from usb
Result: DUT boots and gets to the screen reading: Press enter to configure
* Hit the Enter key
Result: DUT moves to: Ubuntu Core Configure the network and setup an administrator account on this all-sanp Ubuntu Core system
* Hit the enter key
Result: DUT moves to: Network Connections Configure at least the main interface this server will use to receive updates <Network interfaces>
* Move the highlight up to the wlan0 and tap enter
Result: DUT moves to Network interface wlan0
* Move the highlight to choose a visible network and tap enter
Result: Select a network popup appears
* Move up the highlight to network name field and edit it. Add or remove characters for example.
Result: Network name edited is displayed
* Move the highlight to select the network to connect to and tap enter
Result: Popup disappears and the name of the ap is in the main Network interface wlan0 manual IPv4 configuration page
* If required highlight the password, add the password and move to done and hit enter
Result: Password is added and the DUT moves to: Network Interface wlan0
* Move the highlight to done and hit enter
Result: DUT moves to: Network connections
* With the network setup move the highlight to done and hit enter
Result: DUT displays: "Network configuration timed out; please verify your settings." This message proves that our wlan0 config is wrong and there is no internet access. DUT shouldn't move to profile setup page in any case.