v2019dev0/TestReports/20190412/OSTree

From Apertis
Jump to: navigation, search


Testing results considerations:

  1. These tests use OSTree images.
  2. Manual test cases were executed on target Arm, target AMD64 and SDK AMD64 images when applicable.
  3. All automated test cases already integrated into LAVA system were executed on the following images: target Arm, target AMD64.
  4. Failures and partial failures were filed as bugs in Phabricator for further investigation and analysis.



The platform image used for these tests are:

Automated tests

Automated tests results are available at http://lavaphabbridge.apertis.org/report/v2019dev0/20190412.1/ostree

Manual tests

ID Description Type Minimal Arm Minimal SabreLite Minimal Arm64 Minimal AMD64 Target Arm Target AMD64 Tested By Notes
Sanity
sanity-check Check that the Apertis system basically works. (manual) Manual PASSED PASSED PASSED PASSED PASSED PASSED sagar
Booting
image-bootable Test whether the image is bootable. Manual PASSED PASSED PASSED PASSED PASSED PASSED sagar
image-gui-start Test whether the image starts a graphical environment. Manual N/A N/A N/A N/A PASSED PASSED sagar


Multitouch
webkit2gtk-mt-touch-events Test implementation of W3C touch events API in WebKit2GTK. Manual N/A N/A N/A N/A PASSED PASSED sagar
WebKit
webkit2gtk-actor-tiles Test implementation of actor-based tiled backing store in webkit2GTK. Manual N/A N/A N/A N/A NOT TESTED NOT TESTED sagar test is deprecated
webkit2gtk-event-handling-redesign Test WebKit2Gtk ability to scroll by touch with inertial behaviour after the touch ended. Manual N/A N/A N/A N/A PARTIAL FAILED sagar T4243 T6012
Webkit2GTK-AC
webkit2gtk-ac-3d-rendering Test that 3D transforms are applied correctly. Manual N/A N/A N/A N/A PASSED PASSED sagar
webkit2gtk-ac-animations Test that Webkit2GTK actors created for representing layers animate properly. Manual N/A N/A N/A N/A PASSED PASSED sagar
webkit2gtk-ac-scrolling Test that actors created for representing layers scroll properly. Manual N/A N/A N/A N/A PASSED PASSED sagar
Meta Data Management


tracker-indexing-mass-storage Check tracker indexes mass storage devices on insertion. Manual N/A N/A N/A N/A FAILED FAILED sagar T5734


Networking
bluez-avrcp-volume Test the AVRCP volume up and down commands. Manual N/A N/A N/A N/A FAILED FAILED sagar T5734
bluez-phone Test all BlueZ features needing interaction with a phone (Pairing, OBEX, PBAP, AVRCP, A2DP, HSP, HFP, PAN). Manual N/A N/A N/A N/A FAILED PASSED sagar T5804
bluez-setup Test setup of Bluetooth stack by running basic tests using hciconfig. Check if the interface can be brought up/down, can be changed of mode and can discover devices. Manual N/A N/A N/A N/A PASSED PASSED sagar
connman-pan-tethering Test ConnMan support for PAN connectivity. Manual N/A N/A N/A N/A FAILED FAILED sagar T5929
connman-pan-network-access Test ConnMan support for bluetooth PAN Network Access Profile. Manual N/A N/A N/A N/A FAILED FAILED sagar T5930
connman-services Tests ConnMan services. Manual N/A N/A N/A N/A PASSED PASSED sagar
connman-usb-tethering Test ConnMan support for USB tethering (connection side). Manual N/A N/A N/A N/A FAILED FAILED sagar T5931
ofono-sms-receive Test receiving SMS messages with oFono. Manual N/A N/A N/A N/A FAILED FAILED sagar T5734
ofono-sms-send Test sending of SMS with oFono. Manual N/A N/A N/A N/A FAILED FAILED sagar T5734
Contacts
evolution-sync-bluetooth Synchronizing Bluetooth-paired phone contacts to Evolution. Manual N/A N/A N/A N/A FAILED FAILED sagar T5734
Connectivity
bluez-hfp Test Hands Free Profile (HFP) BlueZ feature. Manual N/A N/A N/A N/A FAILED FAILED sagar T5734
rfkill-toggle Test that rfkill is functional. Manual N/A N/A N/A N/A PASSED PASSED sagar
telepathy-ring Support making and receiving phone calls in Telepathy manually, with a bluetooth-paired phone. Manual N/A N/A N/A N/A FAILED FAILED sagar T5734
Contacts
folks-ofono-backend Reading contacts from 3G modem SIM card. Manual N/A N/A N/A N/A FAILED FAILED sagar T5734
Multimedia
webkit2gtk-gstreamer1.0 Tests html5 video playback with GStreamer 1.0 in WebKit2GTK Manual N/A N/A N/A N/A FAILED PASSED sagar T5906
traffic-control-basic Test basic traffic control support. This test does not use tcmmd but it could catch missing support in the kernel or misbehaving traffic control. Manual N/A N/A N/A N/A FAILED FAILED sagar T5734
video-animation-on-boot Test whether Plymouth is able to display a graphical theme. Manual N/A N/A N/A N/A FAILED FAILED sagar T5798
Security
apparmor-bluez Test AppArmor profile for support for SPP profile (bluez-spp) and core functionalities (bluez-core). Manual N/A N/A N/A N/A PASSED PASSED sagar
apparmor-bluez-avrcp-volume Test AppArmor profile for Bluez AVRCP volume up and down commands. Manual N/A N/A N/A N/A FAILED FAILED sagar T5734
apparmor-bluez-setup Test AppArmor profiles for setting up of Bluetooth stack by running basic tests using hciconfig. Check if the interface can be brought up/down, can be changed of mode and can discover devices. Manual N/A N/A N/A N/A PASSED PASSED sagar T5803
apparmor-ofono Test that AppArmor profiles provided in ofono package work without unexpected auditd log entries. Manual N/A N/A N/A N/A FAILED FAILED sagar T5734
iptables-nmap Test the firewall using nmap from another computer. Manual PASSED PASSED PASSED PASSED PASSED PASSED sagar
Webkit
webkit2gtk-aligned-scroll Test that scrolling is pinned in a given direction when started mostly towards it. Manual N/A N/A N/A N/A NOT TESTED NOT TESTED sagar test deprecated
webkit2gtk-lazy-click Test touch adjustment (lazy click) support in WebKit2GTK Manual N/A N/A N/A N/A NOT TESTED NOT TESTED sagar test deprecated
webkit2gtk-drag-and-drop Test HTML5 drag-and-drop support in WebKit2GTK Manual N/A N/A N/A N/A PASSED PASSED sagar
webkit2gtk-contextual-zoom This test case is about zooming parts of a web page to make it easier to read on devices with limited screen area. The scope of the test is the zooming functionality that takes into account the content in the clicked area, other functionality that belongs to a real browser isn't to be tested here. Manual N/A N/A N/A N/A NOT TESTED NOT TESTED sagar test deprecated
Apertis update manager
apertis-update-manager-api Test the apertis-update-manager automatic update. Manual N/A FAILED N/A N/A N/A N/A sagar T5903
apertis-update-manager-auto Test the apertis-update-manager automatic update via mass storage device. Manual N/A FAILED N/A N/A N/A N/A sagar T5885
apertis-update-manager-diskfull Ensure that failures due to disk full errors during updates have no effect on the system, Manual N/A FAILED N/A N/A N/A N/A sagar
apertis-update-manager-manual Test the manual update with apertis-update-manager. Manual N/A PASSED N/A N/A N/A N/A sagar T5907
apertis-update-manager-powercut Ensure that failures due to power losses during updates have no effect on the system. Manual N/A PASSED N/A N/A N/A N/A sagar
apertis-update-manager-rollback Test the automatic rollback and blacklist mechanism of apertis-update-manager. Manual N/A FAILED N/A N/A N/A N/A sagar T5904
apertis-update-manager-usb-unplug Test the apertis-update-manager automatic update interrupted by mass storage removing. Manual N/A FAILED N/A N/A N/A N/A sagar T5908
PASSED - All tests passed.
Automated PARTIAL - Tests mostly passed, with some minor problems.
. FAILED - One or more tests failed.
NOT TESTED - Test not run due to some issue, like WIP, Disabled, lacking proper hardware.
CRITICAL test case - release blocker
HIGH priority test case - must be tested every week
MEDIUM priority test case - must be tested at least every other week
LOW priority test case - test when possible
Personal tools
Namespaces

Variants
Actions
Navigation
Tools