Amplitude
How do I get started?
- Login to our workspace (named
Sourcegraph
). If you don’t have an account, post in #it-tech-ops or request one during login. - Watch this short walkthrough by @ericbm with a couple of members of our marketing team. It goes over the data in Amplitude and how to build many different kinds of analyses. You can also read the Amplitude docs for building analyses and/or check out some of our tutorials.
- Look at all the Team Spaces and join whichever ones are relevant to you. Learn more about Team Spaces here.
- Read the Sourcegraph <> Amplitude FAQs, Amplitude help center or Ask BizOps for help if you have any questions. As a reminder, you can always post in #analytics-review if you have a work-in-progress analysis you want someone else’s eyes on.
Why are we using Amplitude?
Amplitude is a product analytics tool specializing in turning event data into actionable insights and dashboards. Any question about how clients use our product is probably easiest to answer in Amplitude.
Why aren’t we using Looker for this?
Looker is very flexible in that we can set it up for any purpose we’d like. The downside is that to provide a great self-service experience for questions about product usage, setting up Looker would take a lot of work. We get these capabilities out of the box with Amplitude.
What is in Looker vs. Amplitude?
Anything not based directly on analyzing Sourcegraph Cloud events is in Looker. This includes pings from on-prem instances, anything we get from the Cloud Postgres database and any data from third-parties tools (such as Google Analytics and Salesforce).
Type of analysis | Tool | Example |
---|---|---|
Retention/engagement | Amplitude | Link |
Adoption of public code | Looker | Link |
Progress towards OKRs | Looker | Link |
On-prem instances/pings | Looker | Link |
Signup funnel on Cloud | Amplitude | Link |
Any analysis we conducted in Looker before we started using Amplitude we can continue doing in Looker. We’ll still maintain existing Looker dashboards and visualizations. Amplitude will help us conduct new and different analyses regarding product analytics.
Data
Overview
Most Sourcegraph Cloud events are being sent to Amplitude. The events not being sent are extremely low traffic events, non-UI events (e.g. backend events for a search that we capture) or ones we have explicitly decided to exclude (such as code insights events because we’re focused on enterprise and Cloud data won’t inform any decisions). The full data map is in Drive.
The data is currently backfilled from .
It’s sent through a script that runs every hour and pulls from sourcegraph_analytics.amplitude_events_v5
, which is loaded by this scheduled query.
Adding event properties
Event properties are attributes of a particular event. These are added to the amplitude_events_v5
table as individual columns from this scheduled query. For example, here’s a snippet that extracts event properties from both the argument field of an event as well as an event itself.
The steps to adding additional event properties to the query/table:
- Create a PR to update the
amplitude_events_v5
scheduled query with the new event properties. - Once another team member approves, open a blank BigQuery query editor, run the query, and then save the results to a new test table (for example,
amplitude_test_20210812
). - Create an issue for Data Engineering to upload a sample of the new test table to a test project in Amplitude.
- If the data shows up as expected in Amplitude, go ahead and update the
amplitude_events_v5
query with your PR. - Update the INSERT statement with the new event properties. You can’t backfill event properties to events that already exist in Amplitude; however, you can backfill event properties if you’re backfilling events also.
If you’re adding a new event property, please add it to the data map.
JSON_EXTRACT(argument,
'$.code_search.results.results_count') AS results_count,
JSON_EXTRACT(argument,
'$.code_search.results.alert') AS alert,
CASE
WHEN name = 'ViewRepogroup:python' THEN 'python'
WHEN name = 'ViewRepogroup:golang' THEN 'golang'
WHEN name = 'ViewRepogroup:android' THEN 'android'
WHEN name = 'ViewRepogroup:kubernetes' THEN 'kubernetes'
WHEN name = 'ViewRepogroup:kubernetes' THEN 'kubernetes'
WHEN name = 'ViewRepogroup:kubernetes' THEN 'kubernetes'
WHEN name = 'ViewRepogroup:stanford' THEN 'stanford'
WHEN name = 'ViewRepogroup:cncf' THEN 'cncf'
ELSE
NULL
END
repogroup_name,
Adding user properties
User properties are the attributes of individual users. The amplitude_user_characteristics
table defines all user properties, and we join in this table with every event that we pass to Amplitude. These user properties include every A/B test from ab_test_users
. See more in the A/B testing doc (coming soon).
If an additional user property is added to this table, it will only be applied to events and users going forward. User properties are sent to Amplitude along with events, so new properties will only be added when a user triggers an event that is sent to Amplitude. When adding a user property, please add it to the data map
Adding events to Amplitude
We’ve built Amplitude on top of our existing eventLogger infrastructure, so we consider an event to be anything logged by this. We use object action framework in Proper Case for our naming. For example, in eventLogger, this would show up as ‘SearchSubmitted’, and in Amplitude, it’s converted to ‘Search Submitted’.
Namespace your event names in a way that events in a group are easily recognized. For example, the name ClickedCTA
follows the naming convention but doesn’t help users distinguish which CTA was triggered or where the event occurred. PostSignUpFlowClickedBextCTA
helps identify this CTA was for the browser extension.
We send all events from eventLogger to Amplitude except if explicitly added to a deny list in the scheduled query.
Amplitude Govern
Amplitude Govern allows us to manage our data within Amplitude itself (opposed to within data pipelines). The ways we currently use Govern are:
- Any new event is blocked from being visible in Amplitude until it’s adapted (if necessary) to fit the following Amplitude “rules” and then unblocked by someone on the DataOps team
- The event name is converted to object action framework in Proper Case
- The event is set to active/inactive to follow our Cloud active user definitions
- The event is grouped in the appropriate category (such as
Search and navigation
orCode intel
)
- You can merge events if they represent the same action. For example, we deprecated an event representing a sign-up in favor of a new event. In Govern, we can merge these two events to continue tracking sign-up events over time.
A/B testing in Amplitude
Each A/B test has a user property where the A/B test is true
or false
(true
= they saw the variant, false
= they saw the original). See the page on A/B testing for more information about experimentation at Sourcegraph.
Using Amplitude FAQs
How do I find what we call an event in the Sourcegraph code?
Most often, you can search in the ‘Select event’ dropdown, and you’ll find the event you’re looking for. If you can’t or are unsure what the Amplitude event corresponds to, keep reading!
To figure out an event’s name is in code or to confirm whether an existing event in Amplitude is the correct one, follow these steps (or watch this video) :
To find what a button, page view, or action is named
- Open Safari or Chrome
- Enable developer mode (preferences -> advanced)
- Go to sourcegraph.com
- Show the JavaScript console
- Enter
localStorage.eventLogDebug = "true"
- Start using Sourcegraph and click/view the event you’re looking for!
Where should I save charts?
There are three levels to where you can save a chart:
- To create a private chart, uncheck ‘Make Discoverable’ in the chart’s dropdown (“More”).
- To create a chart that is discoverable by other teammates but not as easily accessible unless they search (or you send them a link), just create a chart. This is Amplitude’s default.
- Add charts to Team Spaces if you want them to be easily accessible to teammates.
What are Team Spaces?
Team Spaces are a collection of charts, dashboards and notebooks that are relevant to a specific group (Amplitude’s version of a folder system). By joining one, Amplitude will notify you of every new analysis that a teammate adds to the space (and they’ll be notified of yours, too). You can see all of our Team Spaces here.