Snap metrics: How are “unique” devices counted?

[tobias]

Quick question regarding snap metrics in the dashboard: I read this thread

What does "Daily active devices" in the Snapcraft dashboard mean?

The Snapcraft Stats page says "Daily active devices" tracks the number of unique devices that have contacted the Snap Store using a 7 day window, broken down by channel. That would be Weekly Active Users (WAU), which is a different metric. It would be cool to have both DAU and WAU like in the Apple and Google stores, but if I had to choose, I would go with DAU because it’s more immediately useful following releases. Is the description trying to say that the user metric service only reports ap…

… and would like to ask for some clarification on two details of this metric (since called “Weekly Active Devices”):

1. What counts as “unique”, especially in the context of Ubuntu Core? Most of our users download a ready-made custom image, which (at the moment) does not include any unique identifier. The image contains a local user, so there’s no snap store login unless a user explicitly logs in with their own account.

2. Which “contacts” to the Snap Store are counted? Only during a snap refresh, or is there some other contact ping running in Ubuntu Core? I ask because our image has a pre-configured refresh window between 3-5am, and some users switch off their devices overnight.

Thanks in advance!
(ping @cprov)

[noise]

@tobias

What counts as “unique”, especially in the context of Ubuntu Core? Most of our users download a ready-made custom image, which (at the moment) does not include any unique identifier. The image contains a local user, so there’s no snap store login unless a user explicitly logs in with their own account.

Assuming your image is setup to do a normal device-registration flow on first boot to obtain a serial assertion then that data (brand, model, serial) is the ID used for active device metrics.

@tobias

Which “contacts” to the Snap Store are counted? Only during a snap refresh, or is there some other contact ping running in Ubuntu Core? I ask because our image has a pre-configured refresh window between 3-5am, and some users switch off their devices overnight.

Device metrics are derived from refresh requests as you suspect. Snapd will attempt to make at least one refresh call per day, so in your scenario if the device missed its normal refresh window, it would do a no-op refresh call at a time when it is alive that day. If a device is down for longer than 24 hrs, obviously we can’t hear from it and it will not be counted for that period.

Let me know if that clears things up and if you have further questions.

[tobias]

Thank you, that clears it up quite a bit for me!

Assuming your image is setup to do a normal device-registration flow on first boot to obtain a serial assertion then that data (brand, model, serial) is the ID used for active device metrics.

That’s exactly the thing our image is not set up for right now – mainly because we don’t have the proper infrastructure for device-registration on our servers yet (small OSS project with currently one developer – me). I really should’ve left this extra tidbit from my initial draft in the post

Given a devices has no valid serial assertion: Does it count towards the store metric at all? If so, which help metric is used to differentiate devices (e.g. IP addresses which may be inherently error-prone, ofc)?

I’m not sure if that matters in this context, but there are two things to consider in our release flow regarding registration:

1. We “pre-boot” the generated .img from ubuntu-image on a Raspberry Pi, because the initial boot and setup takes well over 10 minutes on a Pi 3B – I know that the main reason for this is out of Canonical’s direct control. During this boot, the screen shows our boot logo until mir-kiosk is installed, a black screen afterwards, and we need to reboot once so that network-manager can take command of the ethernet interface. All not great first impressions, and most users are rather impatient when trying out new stuff (let alone they read “get a coffee during first boot, don’t cut power, takes some time” ).
2. When they flash our pre-booted image and start it up, the Raspberry Pi does not have internet connectivity until they configure credentials for their home WiFi (or set the device to LAN access) through a setup WiFi provided by the Pi. I guess the register-device hook will retry later when it fails due to missing connectivity at first boot, but I haven’t had the time to tinker with it anyway.