Amplitude

Why aren’t we using Looker for this?

Looker is very flexible in that we can set it up for any purpose we’d like. The downside is that to provide a great self-service experience for questions about product usage, setting up Looker would take a lot of work. We get these capabilities out of the box with Amplitude.

What is in Looker vs. Amplitude?

Anything not based directly on analyzing Sourcegraph Cloud events is in Looker. This includes pings from on-prem instances, anything we get from the Cloud Postgres database and any data from third-parties tools (such as Google Analytics and Salesforce).

Any analysis we conducted in Looker before we started using Amplitude we can continue doing in Looker. We’ll still maintain existing Looker dashboards and visualizations. Amplitude will help us conduct new and different analyses regarding product analytics.

Overview

Most Sourcegraph Cloud events are being sent to Amplitude. The events not being sent are extremely low traffic events, non-UI events (e.g. backend events for a search that we capture) or ones we have explicitly decided to exclude (such as code insights events because we’re focused on enterprise and Cloud data won’t inform any decisions). The full data map is in Drive.

The data is currently backfilled from 2021-10-01.

It’s sent through a script that runs every hour and pulls from sourcegraph_analytics.amplitude_events_v5, which is loaded by this scheduled query.

Adding event properties

Event properties are attributes of a particular event. These are added to the amplitude_events_v5 table as individual columns from this scheduled query. For example, here’s a snippet that extracts event properties from both the argument field of an event as well as an event itself.

The steps to adding additional event properties to the query/table:

1. Create a PR to update the amplitude_events_v5 scheduled query with the new event properties.
2. Once another team member approves, open a blank BigQuery query editor, run the query, and then save the results to a new test table (for example, amplitude_test_20210812).
3. Create an issue for Data Engineering to upload a sample of the new test table to a test project in Amplitude.
4. If the data shows up as expected in Amplitude, go ahead and update the amplitude_events_v5 query with your PR.
5. Update the INSERT statement with the new event properties. You can’t backfill event properties to events that already exist in Amplitude; however, you can backfill event properties if you’re backfilling events also.

If you’re adding a new event property, please add it to the data map.

JSON_EXTRACT(argument,
  '$.code_search.results.results_count') AS results_count,
JSON_EXTRACT(argument,
  '$.code_search.results.alert') AS alert,
CASE
  WHEN name = 'ViewRepogroup:python' THEN 'python'
  WHEN name = 'ViewRepogroup:golang' THEN 'golang'
  WHEN name = 'ViewRepogroup:android' THEN 'android'
  WHEN name = 'ViewRepogroup:kubernetes' THEN 'kubernetes'
  WHEN name = 'ViewRepogroup:kubernetes' THEN 'kubernetes'
  WHEN name = 'ViewRepogroup:kubernetes' THEN 'kubernetes'
  WHEN name = 'ViewRepogroup:stanford' THEN 'stanford'
  WHEN name = 'ViewRepogroup:cncf' THEN 'cncf'
ELSE
NULL
END
repogroup_name,

Adding user properties

User properties are the attributes of individual users. The amplitude_user_characteristics table defines all user properties, and we join in this table with every event that we pass to Amplitude. These user properties include every A/B test from ab_test_users. See more in the A/B testing doc (coming soon).

If an additional user property is added to this table, it will only be applied to events and users going forward. User properties are sent to Amplitude along with events, so new properties will only be added when a user triggers an event that is sent to Amplitude. When adding a user property, please add it to the data map

Adding events to Amplitude

We’ve built Amplitude on top of our existing eventLogger infrastructure, so we consider an event to be anything logged by this. We use object action framework in Proper Case for our naming. For example, in eventLogger, this would show up as ‘SearchSubmitted’, and in Amplitude, it’s converted to ‘Search Submitted’.

Namespace your event names in a way that events in a group are easily recognized. For example, the name ClickedCTA follows the naming convention but doesn’t help users distinguish which CTA was triggered or where the event occurred. PostSignUpFlowClickedBextCTA helps identify this CTA was for the browser extension.

We send all events from eventLogger to Amplitude except if explicitly added to a deny list in the scheduled query.

How do I find what we call an event in the Sourcegraph code?

Most often, you can search in the ‘Select event’ dropdown, and you’ll find the event you’re looking for. If you can’t or are unsure what the Amplitude event corresponds to, keep reading!

To figure out an event’s name is in code or to confirm whether an existing event in Amplitude is the correct one, follow these steps (or watch this video) :

To find what a button, page view, or action is named

1. Open Safari or Chrome
2. Enable developer mode (preferences -> advanced)
3. Go to sourcegraph.com
4. Show the JavaScript console
5. Enter localStorage.eventLogDebug = "true"
6. Start using Sourcegraph and click/view the event you’re looking for!

Where should I save charts?

There are three levels to where you can save a chart:

1. To create a private chart, uncheck ‘Make Discoverable’ in the chart’s dropdown (“More”).
2. To create a chart that is discoverable by other teammates but not as easily accessible unless they search (or you send them a link), just create a chart. This is Amplitude’s default.
3. Add charts to Team Spaces if you want them to be easily accessible to teammates.

What are Team Spaces?

Team Spaces are a collection of charts, dashboards and notebooks that are relevant to a specific group (Amplitude’s version of a folder system). By joining one, Amplitude will notify you of every new analysis that a teammate adds to the space (and they’ll be notified of yours, too). You can see all of our Team Spaces here.