v2019dev0/TestReports/20190401

From Apertis
Jump to: navigation, search


Testing results considerations:

  1. Manual test cases were executed on target Arm, target AMD64, base SDK AMD64 and SDK AMD64 images when applicable.
  2. All automated test cases already integrated into LAVA system were executed on the following images: target Arm, target AMD64, development AMD64, base SDK AMD64 and SDK AMD64.
  3. Failures and partial failures were filed as bugs in Phabricator for further investigation and analysis.

For OSTree tests, please refer to V2019dev0/TestReports/20190401/OSTree.

For tiny-containers tests, please refer to V2019dev0/TestReports/20190401/tiny-containers.


The platform image used for these tests are:

Automated tests

Automated tests results are available at http://lavaphabbridge.apertis.org/report/v2019dev0/20190401.0

Manual tests

ID Description Type Minimal Arm Minimal SabreLite Minimal Arm64 Minimal AMD64 Target Arm Target AMD64 Base SDK AMD64 SDK AMD64 Tested By Notes
Sanity
sanity-check Check that the Apertis system basically works. (manual) Manual PASSED PASSED PASSED PASSED PASSED PASSED PASSED sagar
Booting
image-bootable Test whether the image is bootable. Manual PASSED PASSED PASSED PASSED PASSED PASSED PASSED sagar
image-gui-start Test whether the image starts a graphical environment. Manual N/A N/A N/A N/A PASSED PASSED PASSED sagar
HW Acceleration
x-hw-accelerated Test whether the correct GLES2 render is used. Manual N/A N/A N/A N/A N/A N/A
Multitouch
webkit2gtk-mt-touch-events Test implementation of W3C touch events API in WebKit2GTK. Manual N/A N/A N/A N/A
WebKit
webkit2gtk-actor-tiles Test implementation of actor-based tiled backing store in webkit2GTK. Manual N/A N/A N/A N/A
webkit2gtk-event-handling-redesign Test WebKit2Gtk ability to scroll by touch with inertial behaviour after the touch ended. Manual N/A N/A N/A N/A
Webkit2GTK-AC
webkit2gtk-ac-3d-rendering Test that 3D transforms are applied correctly. Manual N/A N/A N/A N/A
webkit2gtk-ac-animations Test that Webkit2GTK actors created for representing layers animate properly. Manual N/A N/A N/A N/A
webkit2gtk-ac-scrolling Test that actors created for representing layers scroll properly. Manual N/A N/A N/A N/A
Meta Data Management
tracker-indexing-mass-storage Check tracker indexes mass storage devices on insertion. Manual N/A N/A N/A N/A
Networking
bluez-avrcp-volume Test the AVRCP volume up and down commands. Manual N/A N/A N/A N/A
bluez-phone Test all BlueZ features needing interaction with a phone (Pairing, OBEX, PBAP, AVRCP, A2DP, HSP, HFP, PAN). Manual N/A N/A N/A N/A PASSED FAILED FAILED SAGAR T5807T5804
bluez-setup Test setup of Bluetooth stack by running basic tests using hciconfig. Check if the interface can be brought up/down, can be changed of mode and can discover devices. Manual N/A N/A N/A N/A PASSED FAILED PASSED SAGAR T5806
connman-pan-tethering Test ConnMan support for PAN connectivity. Manual N/A N/A N/A N/A
connman-pan-network-access Test ConnMan support for bluetooth PAN Network Access Profile. Manual N/A N/A N/A N/A
connman-services Tests ConnMan services. Manual N/A N/A N/A N/A PASSED PASSED PASSED sagar
connman-usb-tethering Test ConnMan support for USB tethering (connection side). Manual N/A N/A N/A N/A
ofono-sms-receive Test receiving SMS messages with oFono. Manual N/A N/A N/A N/A
ofono-sms-send Test sending of SMS with oFono. Manual N/A N/A N/A N/A
SDK Development
sdk-ide-build-run-program Creates a simple test project and confirms that compiling and executing work. Manual N/A N/A N/A N/A N/A N/A N/A
sdk-ade-remote-debugging Checks ade debug remote debugging functionality by connecting to a remote target running gdbserver Manual N/A N/A N/A N/A N/A N/A
sdk-dbus-tools-bustle Test that bustle-dbus-monitor shows services on the session bus. Manual N/A N/A N/A N/A N/A N/A
sdk-dbus-tools-d-feet Test that D-Feet both shows services on the session bus and that new services appear automatically. Manual N/A N/A N/A N/A N/A N/A
sdk-vb-fullscreen Tests that the SDK VM can be fullscreened in VirtualBox Manual N/A N/A N/A N/A N/A N/A FAILED FAILED sagar T5800
Eclipse
eclipse-sysroot-management Test sysroot management and update within Eclipse. It will go through configuring an existing sysroot in Eclipse, and then updating it from the web with the latest version. Manual N/A N/A N/A N/A N/A N/A N/A
eclipse-plugins-install-target Test copying files to the remote device from within Eclipse. It will create a dummy project and then create a configuration to copy the binary file to a remote device and debug it remotely using Eclipse. Manual N/A N/A N/A N/A N/A N/A N/A
eclipse-remote-debugging Test remote debugging within Eclipse. It will create a dummy project and then create a configuration to copy the binary file to a remote device and debug it remotely using Eclipse. Manual N/A N/A N/A N/A N/A N/A N/A
Contacts
evolution-sync-bluetooth Synchronizing Bluetooth-paired phone contacts to Evolution. Manual N/A N/A N/A N/A N/A
Connectivity
bluez-hfp Test Hands Free Profile (HFP) BlueZ feature. Manual N/A N/A N/A N/A
rfkill-toggle Test that rfkill is functional. Manual N/A N/A N/A N/A N/A N/A
telepathy-ring Support making and receiving phone calls in Telepathy manually, with a bluetooth-paired phone. Manual N/A N/A N/A N/A N/A
Contacts
folks-ofono-backend Reading contacts from 3G modem SIM card. Manual N/A N/A N/A N/A N/A
Multimedia
webkit2gtk-gstreamer1.0 Tests html5 video playback with GStreamer 1.0 in WebKit2GTK Manual N/A N/A N/A N/A
traffic-control-basic Test basic traffic control support. This test does not use tcmmd but it could catch missing support in the kernel or misbehaving traffic control. Manual N/A N/A N/A N/A
video-animation-on-boot Test whether Plymouth is able to display a graphical theme. Manual N/A N/A N/A N/A FAILED FAILED FAILED SAGAR T5798
Security
apparmor-bluez Test AppArmor profile for support for SPP profile (bluez-spp) and core functionalities (bluez-core). Manual N/A N/A N/A N/A PARTIAL N/A N/A SAGAR T5823
apparmor-bluez-avrcp-volume Test AppArmor profile for Bluez AVRCP volume up and down commands. Manual N/A N/A N/A N/A N/A N/A
apparmor-bluez-setup Test AppArmor profiles for setting up of Bluetooth stack by running basic tests using hciconfig. Check if the interface can be brought up/down, can be changed of mode and can discover devices. Manual N/A N/A N/A N/A PASSED N/A N/A sagar
apparmor-ofono Test that AppArmor profiles provided in ofono package work without unexpected auditd log entries. Manual N/A N/A N/A N/A N/A N/A
iptables-nmap Test the firewall using nmap from another computer. Manual N/A N/A N/A N/A PASSED PASSED PASSED sagar
Webkit
webkit2gtk-aligned-scroll Test that scrolling is pinned in a given direction when started mostly towards it. Manual N/A N/A N/A N/A N/A N/A
webkit2gtk-lazy-click Test touch adjustment (lazy click) support in WebKit2GTK Manual N/A N/A N/A N/A
webkit2gtk-drag-and-drop Test HTML5 drag-and-drop support in WebKit2GTK Manual N/A N/A N/A N/A
webkit2gtk-contextual-zoom This test case is about zooming parts of a web page to make it easier to read on devices with limited screen area. The scope of the test is the zooming functionality that takes into account the content in the clicked area, other functionality that belongs to a real browser isn't to be tested here. Manual N/A N/A N/A N/A
PASSED - All tests passed.
PARTIAL - Tests mostly passed, with some minor problems.
FAILED - One or more tests failed.
NOT TESTED - Test not run due to some issue, like WIP, Disabled, lacking proper hardware.
CRITICAL test case - release blocker
HIGH priority test case - must be tested every week
MEDIUM priority test case - must be tested at least every other week
LOW priority test case - test when possible
Personal tools
Namespaces

Variants
Actions
Navigation
Tools