Usage Ping Guide

Version history
  • Introduced in GitLab Enterprise Edition 8.10.
  • More statistics were added in GitLab Enterprise Edition 8.12.
  • Moved to GitLab Core in 9.1.
  • More statistics were added in GitLab Ultimate 11.2.

This guide describes Usage Ping’s purpose and how it’s implemented.

For more information about Telemetry, see:

More useful links:

What is Usage Ping?

  • GitLab sends a weekly payload containing usage data to GitLab Inc. Usage Ping provides high-level data to help our product, support, and sales teams. It does not send any project names, usernames, or any other specific data. The information from the usage ping is not anonymous, it is linked to the hostname of the instance. Sending usage ping is optional, and any instance can disable analytics.
  • The usage data is primarily composed of row counts for different tables in the instance’s database. By comparing these counts month over month (or week over week), we can get a rough sense for how an instance is using the different features within the product. In addition to counts, other facts that help us classify and understand GitLab installations are collected.
  • Usage ping is important to GitLab as we use it to calculate our Stage Monthly Active Users (SMAU) which helps us measure the success of our stages and features.
  • Once usage ping is enabled, GitLab will gather data from the other instances and will be able to show usage statistics of your instance to your users.

Why should we enable Usage Ping?

  • The main purpose of Usage Ping is to build a better GitLab. Data about how GitLab is used is collected to better understand feature/stage adoption and usage, which helps us understand how GitLab is adding value and helps our team better understand the reasons why people use GitLab and with this knowledge we’re able to make better product decisions.
  • As a benefit of having the usage ping active, GitLab lets you analyze the users’ activities over time of your GitLab installation.
  • As a benefit of having the usage ping active, GitLab provides you with The DevOps Score,which gives you an overview of your entire instance’s adoption of Concurrent DevOps from planning to monitoring.
  • You will get better, more proactive support. (assuming that our TAMs and support organization used the data to deliver more value)
  • You will get insight and advice into how to get the most value out of your investment in GitLab. Wouldn’t you want to know that a number of features or values are not being adopted in your organization?
  • You get a report that illustrates how you compare against other similar organizations (anonymized), with specific advice and recommendations on how to improve your DevOps processes.
  • Usage Ping is enabled by default. To disable it, see Disable Usage Ping.

Limitations

  • Usage Ping does not track frontend events things like page views, link clicks, or user sessions, and only focuses on aggregated backend events.
  • Because of these limitations we recommend instrumenting your products with Snowplow for more detailed analytics on GitLab.com and use Usage Ping to track aggregated backend events on self-managed.

Usage Ping payload

You can view the exact JSON payload sent to GitLab Inc. in the administration panel. To view the payload:

  1. Navigate to Admin Area > Settings > Metrics and profiling.
  2. Expand the Usage statistics section.
  3. Click the Preview payload button.

For an example payload, see Example Usage Ping payload.

Disable Usage Ping

To disable Usage Ping in the GitLab UI, go to the Settings page of your administration panel and uncheck the Usage Ping checkbox.

To disable Usage Ping and prevent it from being configured in the future through the administration panel, Omnibus installs can set the following in gitlab.rb:

gitlab_rails['usage_ping_enabled'] = false

Source installations can set the following in gitlab.yml:

production: &base
  # ...
  gitlab:
    # ...
    usage_ping_enabled: false

Usage Ping request flow

The following example shows a basic request/response flow between a GitLab instance, the Versions Application, the License Application, Salesforce, GitLab’s S3 Bucket, GitLab’s Snowflake Data Warehouse, and Sisense:

sequenceDiagram participant GitLab Instance participant Versions Application participant Licenses Application participant Salesforce participant S3 Bucket participant Snowflake DW participant Sisense Dashboards GitLab Instance->>Versions Application: Send Usage Ping loop Process usage data Versions Application->>Versions Application: Parse usage data Versions Application->>Versions Application: Write to database Versions Application->>Versions Application: Update license ping time end loop Process data for Salesforce Versions Application-xLicenses Application: Request Zuora subscription id Licenses Application-xVersions Application: Zuora subscription id Versions Application-xSalesforce: Request Zuora account id by Zuora subscription id Salesforce-xVersions Application: Zuora account id Versions Application-xSalesforce: Usage data for the Zuora account end Versions Application->>S3 Bucket: Export Versions database S3 Bucket->>Snowflake DW: Import data Snowflake DW->>Snowflake DW: Transform data using dbt Snowflake DW->>Sisense Dashboards: Data available for querying Versions Application->>GitLab Instance: DevOps Score (Conversational Development Index)

How Usage Ping works

  1. The Usage Ping cron job is set in Sidekiq to run weekly.
  2. When the cron job runs, it calls GitLab::UsageData.to_json.
  3. GitLab::UsageData.to_json cascades down to ~400+ other counter method calls.
  4. The response of all methods calls are merged together into a single JSON payload in GitLab::UsageData.to_json.
  5. The JSON payload is then posted to the Versions application.

Implementing Usage Ping

Usage Ping consists of two kinds of data, counters and observations. Counters track how often a certain event happened over time, such as how many CI pipelines have run. They are monotonic and always trend up. Observations are facts collected from one or more GitLab instances and can carry arbitrary data. There are no general guidelines around how to collect those, due to the individual nature of that data.

There are four types of counters which are all found in usage_data.rb:

  • Ordinary Batch Counters: Simple count of a given ActiveRecord_Relation
  • Distinct Batch Counters: Distinct count of a given ActiveRecord_Relation on given column
  • Alternative Counters: Used for settings and configurations
  • Redis Counters: Used for in-memory counts. This method is being deprecated due to data inaccuracies and will be replaced with a persistent method.
Note: Only use the provided counter methods. Each counter method contains a built in fail safe to isolate each counter to avoid breaking the entire Usage Ping.

Why batch counting

For large tables, PostgreSQL can take a long time to count rows due to MVCC (Multi-version Concurrency Control). Batch counting is a counting method where a single large query is broken into multiple smaller queries. For example, instead of a single query querying 1,000,000 records, with batch counting, you can execute 100 queries of 10,000 records each. Batch counting is useful for avoiding database timeouts as each batch query is significantly shorter than one single long running query.

For GitLab.com, there are extremely large tables with 15 second query timeouts, so we use batch counting to avoid encountering timeouts. Here are the sizes of some GitLab.com tables:

Table Row counts in millions
merge_request_diff_commits 2280
ci_build_trace_sections 1764
merge_request_diff_files 1082
events 514

There are two batch counting methods provided, Ordinary Batch Counters and Distinct Batch Counters. Batch counting requires indexes on columns to calculate max, min, and range queries. In some cases, a specialized index may need to be added on the columns involved in a counter.

Ordinary Batch Counters

Handles ActiveRecord::StatementInvalid error

Simple count of a given ActiveRecord_Relation

Method: count(relation, column = nil, batch: true, start: nil, finish: nil)

Arguments:

  • relation the ActiveRecord_Relation to perform the count
  • column the column to perform the count on, by default is the primary key
  • batch: default true in order to use batch counting
  • start: custom start of the batch counting in order to avoid complex min calculations
  • end: custom end of the batch counting in order to avoid complex min calculations

Examples:

count(User.active)
count(::Clusters::Cluster.aws_installed.enabled, :cluster_id)
count(::Clusters::Cluster.aws_installed.enabled, :cluster_id, start: ::Clusters::Cluster.minimum(:id), finish: ::Clusters::Cluster.maximum(:id))

Distinct Batch Counters

Handles ActiveRecord::StatementInvalid error

Distinct count of a given ActiveRecord_Relation on given column

Method: distinct_count(relation, column = nil, batch: true, start: nil, finish: nil)

Arguments:

  • relation the ActiveRecord_Relation to perform the count
  • column the column to perform the distinct count, by default is the primary key
  • batch: default true in order to use batch counting
  • start: custom start of the batch counting in order to avoid complex min calculations
  • end: custom end of the batch counting in order to avoid complex min calculations

Examples:

distinct_count(::Project, :creator_id)
distinct_count(::Note.with_suggestions.where(time_period), :author_id, start: ::User.minimum(:id), finish: ::User.maximum(:id))
distinct_count(::Clusters::Applications::CertManager.where(time_period).available.joins(:cluster), 'clusters.user_id')

Redis Counters

Handles ::Redis::CommandError and Gitlab::UsageDataCounters::BaseCounter::UnknownEvent returns -1 when a block is sent or hash with all values -1 when a counter(Gitlab::UsageDataCounters) is sent different behavior due to 2 different implementations of Redis counter

Method: redis_usage_data(counter, &block)

Arguments:

  • counter: a counter from Gitlab::UsageDataCounters, that has fallback_totals method implemented
  • or a block: which is evaluated

Example of usage:

redis_usage_data(Gitlab::UsageDataCounters::WikiPageCounter)
redis_usage_data { ::Gitlab::UsageCounters::PodLogs.usage_totals[:total] }

Note that Redis counters are in the process of being deprecated and you should instead try to use Snowplow events instead. We’re in the process of building self-managed event tracking and once this is available, we will convert all Redis counters into Snowplow events.

Alternative Counters

Handles StandardError and fallbacks into -1 this way not all measures fail if we encounter one exception. Mainly used for settings and configurations.

Method: alt_usage_data(value = nil, fallback: -1, &block)

Arguments:

  • value: a simple static value in which case the value is simply returned.
  • or a block: which is evaluated
  • fallback: -1: the common value used for any metrics that are failing.

Example of usage:

alt_usage_data { Gitlab::VERSION }
alt_usage_data { Gitlab::CurrentSettings.uuid }
alt_usage_data(999)

Prometheus Queries

In those cases where operational metrics should be part of Usage Ping, a database or Redis query is unlikely to provide useful data. Instead, Prometheus might be more appropriate, since most of GitLab’s architectural components publish metrics to it that can be queried back, aggregated, and included as usage data.

Note: Prometheus as a data source for Usage Ping is currently only available for single-node Omnibus installations that are running the bundled Prometheus instance.

In order to query Prometheus for metrics, a helper method is available that will yield a fully configured PrometheusClient, given it is available as per the note above:

with_prometheus_client do |client|
  response = client.query('<your query>')
  ...
end

Please refer to the PrometheusClient definition for how to use its API to query for data.

Developing and testing Usage Ping

1. Use your Rails console to manually test counters

# count
Gitlab::UsageData.count(User.active)
Gitlab::UsageData.count(::Clusters::Cluster.aws_installed.enabled, :cluster_id)

# count distinct
Gitlab::UsageData.distinct_count(::Project, :creator_id)
Gitlab::UsageData.distinct_count(::Note.with_suggestions.where(time_period), :author_id, start: ::User.minimum(:id), finish: ::User.maximum(:id))

2. Generate the SQL query

Your Rails console will return the generated SQL queries.

使用例:

pry(main)> Gitlab::UsageData.count(User.active)
   (2.6ms)  SELECT "features"."key" FROM "features"
   (15.3ms)  SELECT MIN("users"."id") FROM "users" WHERE ("users"."state" IN ('active')) AND ("users"."user_type" IS NULL OR "users"."user_type" IN (6, 4))
   (2.4ms)  SELECT MAX("users"."id") FROM "users" WHERE ("users"."state" IN ('active')) AND ("users"."user_type" IS NULL OR "users"."user_type" IN (6, 4))
   (1.9ms)  SELECT COUNT("users"."id") FROM "users" WHERE ("users"."state" IN ('active')) AND ("users"."user_type" IS NULL OR "users"."user_type" IN (6, 4)) AND "users"."id" BETWEEN 1 AND 100000

3. Optimize queries with #database-lab

Paste the SQL query into #database-lab to see how the query performs at scale.

  • #database-lab is a Slack channel which uses a production-sized environment to test your queries.
  • GitLab.com’s production database has a 15 second timeout.
  • Any single query must stay below 1 second execution time with cold caches.
  • Add a specialized index on columns involved to reduce the execution time.

In order to have an understanding of the query’s execution we add in the MR description the following information:

  • For counters that have a time_period test we add information for both cases:
    • time_period = {} for all time periods
    • time_period = { created_at: 28.days.ago..Time.current } for last 28 days period
  • Execution plan and query time before and after optimization
  • Query generated for the index and time
  • Migration output for up and down execution

We also use #database-lab and explain.depesz.com. For more details, see the database review guide.

Examples of query optimization work:

4. Add the metric definition

When adding, changing, or updating metrics, please update the Usage Statistics definition table.

5. Add new metric to Versions Application

Check if new metrics need to be added to the Versions Application. See usage_data schema and usage data parameters accepted. Any metrics added under the counts key are saved in the counts column.

For further details, see the Process to add additional instrumentation to the Usage Ping.

6. Add the feature label

Add the feature label to the Merge Request for new Usage Ping metrics. These are user-facing changes and are part of expanding the Usage Ping feature.

7. Add a changelog file

Ensure you comply with the Changelog entries guide.

8. Ask for a Telemetry Review

On GitLab.com, we have DangerBot setup to monitor Telemetry related files and DangerBot will recommend a Telemetry review. Mention @gitlab-org/growth/telemetry/engineers in your MR for a review.

Optional: Test Prometheus based Usage Ping

If the data submitted includes metrics queried from Prometheus that you would like to inspect and verify, then you need to ensure that a Prometheus server is running locally, and that furthermore the respective GitLab components are exporting metrics to it. If you do not need to test data coming from Prometheus, no further action is necessary, since Usage Ping should degrade gracefully in the absence of a running Prometheus server.

There are currently three kinds of components that may export data to Prometheus, and which are included in Usage Ping:

  • node_exporter - Exports node metrics from the host machine
  • gitlab-exporter - Exports process metrics from various GitLab components
  • various GitLab services such as Sidekiq and the Rails server that export their own metrics

Test with an Omnibus container

This is the recommended approach to test Prometheus based Usage Ping.

The easiest way to verify your changes is to build a new Omnibus image from your code branch via CI, then download the image and run a local container instance:

  1. From your merge request, click on the qa stage, then trigger the package-and-qa job. This job will trigger an Omnibus build in a downstream pipeline of the omnibus-gitlab-mirror project.
  2. In the downstream pipeline, wait for the gitlab-docker job to finish.
  3. Open the job logs and locate the full container name including the version. It will take the following form: registry.gitlab.com/gitlab-org/build/omnibus-gitlab-mirror/gitlab-ee:<VERSION>.
  4. On your local machine, make sure you are logged in to the GitLab Docker registry. You can find the instructions for this in Authenticating to the GitLab Container Registry.
  5. Once logged in, download the new image via docker pull registry.gitlab.com/gitlab-org/build/omnibus-gitlab-mirror/gitlab-ee:<VERSION>
  6. For more information about working with and running Omnibus GitLab containers in Docker, please refer to GitLab Docker images in the Omnibus documentation.

Test with GitLab development toolkits

This is the less recommended approach, since it comes with a number of difficulties when emulating a real GitLab deployment.

The GDK is not currently set up to run a Prometheus server or node_exporter alongside other GitLab components. If you would like to do so, Monitoring the GDK with Prometheus is a good start.

The GCK has limited support for testing Prometheus based Usage Ping. By default, it already comes with a fully configured Prometheus service that is set up to scrape a number of components, but with the following limitations:

  • It does not currently run a gitlab-exporter instance, so several process_* metrics from services such as Gitaly may be missing.
  • While it runs a node_exporter, docker-compose services emulate hosts, meaning that it would normally report itself to not be associated with any of the other services that are running. That is not how node metrics are reported in a production setup, where node_exporter always runs as a process alongside other GitLab components on any given node. From Usage Ping’s perspective none of the node data would therefore appear to be associated to any of the services running, since they all appear to be running on different hosts. To alleviate this problem, the node_exporter in GCK was arbitrarily “assigned” to the web service, meaning only for this service node_* metrics will appear in Usage Ping.

Usage Statistics definitions

Statistic Section Stage Tier Edition 説明
uuid          
hostname          
version          
installation_type          
active_user_count          
recorded_at          
recording_ce_finished_at       CE+EE When the core features were computed
recording_ee_finished_at       EE When the EE-specific features were computed
edition          
license_md5          
license_id          
historical_max_users          
Name licensee        
Email licensee        
Company licensee        
license_user_count          
license_starts_at          
license_expires_at          
license_plan          
license_trial          
assignee_lists counts        
boards counts        
ci_builds counts verify     Unique builds in project
ci_internal_pipelines counts verify     Total pipelines in GitLab repositories
ci_external_pipelines counts verify     Total pipelines in external repositories
ci_pipeline_config_auto_devops counts verify     Total pipelines from an Auto DevOps template
ci_pipeline_config_repository counts verify     Total Pipelines from templates in repository
ci_runners counts verify     Total configured Runners in project
ci_triggers counts verify     Total configured Triggers in project
ci_pipeline_schedules counts verify     Pipeline schedules in GitLab
auto_devops_enabled counts configure     Projects with Auto DevOps template enabled
auto_devops_disabled counts configure     Projects with Auto DevOps template disabled
deploy_keys counts        
deployments counts release     Total deployments
deployments counts_monthly release     Total deployments last 28 days
dast_jobs counts        
successful_deployments counts release     Total successful deployments
successful_deployments counts_monthly release     Total successful deployments last 28 days
failed_deployments counts release     Total failed deployments
failed_deployments counts_monthly release     Total failed deployments last 28 days
environments counts release     Total available and stopped environments
clusters counts configure     Total GitLab Managed clusters both enabled and disabled
clusters_enabled counts configure     Total GitLab Managed clusters currently enabled
project_clusters_enabled counts configure     Total GitLab Managed clusters attached to projects
group_clusters_enabled counts configure     Total GitLab Managed clusters attached to groups
instance_clusters_enabled counts configure     Total GitLab Managed clusters attached to the instance
clusters_disabled counts configure     Total GitLab Managed disabled clusters
project_clusters_disabled counts configure     Total GitLab Managed disabled clusters previously attached to projects
group_clusters_disabled counts configure     Total GitLab Managed disabled clusters previously attached to groups
instance_clusters_disabled counts configure     Total GitLab Managed disabled clusters previously attached to the instance
clusters_platforms_eks counts configure     Total GitLab Managed clusters provisioned with GitLab on AWS EKS
clusters_platforms_gke counts configure     Total GitLab Managed clusters provisioned with GitLab on GCE GKE
clusters_platforms_user counts configure     Total GitLab Managed clusters that are user provisioned
clusters_applications_helm counts configure     Total GitLab Managed clusters with Helm enabled
clusters_applications_ingress counts configure     Total GitLab Managed clusters with Ingress enabled
clusters_applications_cert_managers counts configure     Total GitLab Managed clusters with Cert Manager enabled
clusters_applications_crossplane counts configure     Total GitLab Managed clusters with Crossplane enabled
clusters_applications_prometheus counts configure     Total GitLab Managed clusters with Prometheus enabled
clusters_applications_runner counts configure     Total GitLab Managed clusters with Runner enabled
clusters_applications_knative counts configure     Total GitLab Managed clusters with Knative enabled
clusters_applications_elastic_stack counts configure     Total GitLab Managed clusters with Elastic Stack enabled
clusters_management_project counts configure     Total GitLab Managed clusters with defined cluster management project
in_review_folder counts        
grafana_integrated_projects counts        
groups counts        
issues counts        
issues_created_from_gitlab_error_tracking_ui counts monitor      
issues_with_associated_zoom_link counts monitor      
issues_using_zoom_quick_actions counts monitor      
issues_with_embedded_grafana_charts_approx counts monitor      
issues_with_health_status counts        
keys counts        
label_lists counts        
lfs_objects counts        
milestone_lists counts        
milestones counts        
pages_domains counts release     Total GitLab Pages domains
pool_repositories counts        
projects counts        
projects_imported_from_github counts        
projects_with_repositories_enabled counts        
projects_with_error_tracking_enabled counts monitor      
protected_branches counts        
releases counts release     Unique release tags
remote_mirrors counts        
requirements_created counts        
snippets counts ‘create’   CE+EE  
snippets counts_monthly ‘create’   CE+EE  
personal_snippets counts ‘create’   CE+EE  
personal_snippets counts_monthly ‘create’   CE+EE  
project_snippets counts ‘create’   CE+EE  
project_snippets counts_monthly ‘create’   CE+EE  
suggestions counts        
todos counts        
uploads counts        
web_hooks counts        
projects_alerts_active counts        
projects_asana_active counts        
projects_assembla_active counts        
projects_bamboo_active counts        
projects_bugzilla_active counts        
projects_buildkite_active counts        
projects_campfire_active counts        
projects_custom_issue_tracker_active counts        
projects_discord_active counts        
projects_drone_ci_active counts        
projects_emails_on_push_active counts        
projects_external_wiki_active counts        
projects_flowdock_active counts        
projects_github_active counts        
projects_hangouts_chat_active counts        
projects_hipchat_active counts        
projects_irker_active counts        
projects_jenkins_active counts        
projects_jira_active counts        
projects_mattermost_active counts        
projects_mattermost_slash_commands_active counts        
projects_microsoft_teams_active counts        
projects_packagist_active counts        
projects_pipelines_email_active counts        
projects_pivotaltracker_active counts        
projects_prometheus_active counts        
projects_pushover_active counts        
projects_redmine_active counts        
projects_slack_active counts        
projects_slack_slash_commands_active counts        
projects_teamcity_active counts        
projects_unify_circuit_active counts        
projects_webex_teams_active counts        
projects_youtrack_active counts        
projects_jira_server_active counts        
projects_jira_cloud_active counts        
projects_jira_dvcs_cloud_active counts        
projects_jira_dvcs_server_active counts        
labels counts        
merge_requests counts        
merge_requests_users counts        
notes counts        
wiki_pages_create counts        
wiki_pages_update counts        
wiki_pages_delete counts        
web_ide_commits counts        
web_ide_views counts        
web_ide_merge_requests counts        
web_ide_previews counts        
snippet_comment counts        
commit_comment counts        
merge_request_comment counts        
snippet_create counts        
snippet_update counts        
navbar_searches counts        
cycle_analytics_views counts        
productivity_analytics_views counts        
source_code_pushes counts        
merge_request_create counts        
design_management_designs_create counts        
design_management_designs_update counts        
design_management_designs_delete counts        
licenses_list_views counts        
user_preferences_group_overview_details counts        
user_preferences_group_overview_security_dashboard counts        
ingress_modsecurity_logging counts        
ingress_modsecurity_blocking counts        
ingress_modsecurity_disabled counts        
ingress_modsecurity_not_installed counts        
dependency_list_usages_total counts        
epics counts        
feature_flags counts        
geo_nodes counts geo     Number of sites in a Geo deployment
geo_event_log_max_id counts geo     Number of replication events on a Geo primary
incident_issues counts monitor     Issues created by the alert bot
alert_bot_incident_issues counts monitor     Issues created by the alert bot
incident_labeled_issues counts monitor     Issues with the incident label
issues_created_gitlab_alerts counts monitor     Issues created from alerts by non-alert bot users
issues_created_manually_from_alerts counts monitor     Issues created from alerts by non-alert bot users
issues_created_from_alerts counts monitor     Issues created from Prometheus and alert management alerts
ldap_group_links counts        
ldap_keys counts        
ldap_users counts        
pod_logs_usages_total counts        
projects_enforcing_code_owner_approval counts        
projects_mirrored_with_pipelines_enabled counts release     Projects with repository mirroring enabled
projects_reporting_ci_cd_back_to_github counts verify     Projects with a GitHub service pipeline enabled
projects_with_packages counts package     Projects with package registry configured
projects_with_prometheus_alerts counts monitor     Projects with Prometheus alerting enabled
projects_with_tracing_enabled counts monitor     Projects with tracing enabled
projects_with_alerts_service_enabled counts monitor     Projects with alerting service enabled
template_repositories counts        
container_scanning_jobs counts        
dependency_scanning_jobs counts        
license_management_jobs counts        
sast_jobs counts        
status_page_projects counts monitor     Projects with status page enabled
status_page_issues counts monitor     Issues published to a Status Page
status_page_incident_publishes counts monitor     Cumulative count of usages of publish operation
status_page_incident_unpublishes counts monitor     Cumulative count of usages of unpublish operation
epics_deepest_relationship_level counts        
operations_dashboard_default_dashboard counts monitor     Active users with enabled operations dashboard
operations_dashboard_users_with_projects_added counts monitor     Active users with projects on operations dashboard
container_registry_enabled          
dependency_proxy_enabled          
gitlab_shared_runners_enabled          
gravatar_enabled          
ldap_enabled          
mattermost_enabled          
omniauth_enabled          
prometheus_enabled         Whether the bundled Prometheus is enabled
prometheus_metrics_enabled          
reply_by_email_enabled          
average avg_cycle_analytics - code        
sd avg_cycle_analytics - code        
missing avg_cycle_analytics - code        
average avg_cycle_analytics - test        
sd avg_cycle_analytics - test        
missing avg_cycle_analytics - test        
average avg_cycle_analytics - review        
sd avg_cycle_analytics - review        
missing avg_cycle_analytics - review        
average avg_cycle_analytics - staging        
sd avg_cycle_analytics - staging        
missing avg_cycle_analytics - staging        
average avg_cycle_analytics - production        
sd avg_cycle_analytics - production        
missing avg_cycle_analytics - production        
total avg_cycle_analytics        
g_analytics_contribution analytics_unique_visits manage     Visits to /groups/:group/-/contribution_analytics
g_analytics_insights analytics_unique_visits manage     Visits to /groups/:group/-/insights
g_analytics_issues analytics_unique_visits manage     Visits to /groups/:group/-/issues_analytics
g_analytics_productivity analytics_unique_visits manage     Visits to /groups/:group/-/analytics/productivity_analytics
g_analytics_valuestream analytics_unique_visits manage     Visits to /groups/:group/-/analytics/value_stream_analytics
p_analytics_pipelines analytics_unique_visits manage     Visits to /:group/:project/pipelines/charts
p_analytics_code_reviews analytics_unique_visits manage     Visits to /:group/:project/-/analytics/code_reviews
p_analytics_valuestream analytics_unique_visits manage     Visits to /:group/:project/-/value_stream_analytics
p_analytics_insights analytics_unique_visits manage     Visits to /:group/:project/insights
p_analytics_issues analytics_unique_visits manage     Visits to /:group/:project/-/analytics/issues_analytics
p_analytics_repo analytics_unique_visits manage     Visits to /:group/:project/-/graphs/master/charts
u_analytics_todos analytics_unique_visits manage     Visits to /dashboard/todos
i_analytics_cohorts analytics_unique_visits manage     Visits to /-/instance_statistics/cohorts
i_analytics_dev_ops_score analytics_unique_visits manage     Visits to /-/instance_statistics/dev_ops_score
analytics_unique_visits_for_any_target analytics_unique_visits manage     Visits to any of the pages listed above
clusters_applications_cert_managers usage_activity_by_stage configure   CE+EE Unique clusters with certificate managers enabled
clusters_applications_helm usage_activity_by_stage configure   CE+EE Unique clusters with Helm enabled
clusters_applications_ingress usage_activity_by_stage configure   CE+EE Unique clusters with Ingress enabled
clusters_applications_knative usage_activity_by_stage configure   CE+EE Unique clusters with Knative enabled
clusters_management_project usage_activity_by_stage configure   CE+EE Unique clusters with project management enabled
clusters_disabled usage_activity_by_stage configure   CE+EE Total non-“GitLab Managed clusters”
clusters_enabled usage_activity_by_stage configure   CE+EE Total GitLab Managed clusters
clusters_platforms_gke usage_activity_by_stage configure   CE+EE Unique clusters with Google Cloud installed
clusters_platforms_eks usage_activity_by_stage configure   CE+EE Unique clusters with AWS installed
clusters_platforms_user usage_activity_by_stage configure   CE+EE Unique clusters that are user provided
instance_clusters_disabled usage_activity_by_stage configure   CE+EE Unique clusters disabled on instance
instance_clusters_enabled usage_activity_by_stage configure   CE+EE Unique clusters enabled on instance
group_clusters_disabled usage_activity_by_stage configure   CE+EE Unique clusters disabled on group
group_clusters_enabled usage_activity_by_stage configure   CE+EE Unique clusters enabled on group
project_clusters_disabled usage_activity_by_stage configure   CE+EE Unique clusters disabled on project
project_clusters_enabled usage_activity_by_stage configure   CE+EE Unique clusters enabled on project
projects_slack_notifications_active usage_activity_by_stage configure   EE Unique projects with Slack service enabled
projects_slack_slash_active usage_activity_by_stage configure   EE Unique projects with Slack ‘/’ commands enabled
projects_with_prometheus_alerts usage_activity_by_stage configure   EE Projects with Prometheus enabled and no alerts
deploy_keys usage_activity_by_stage create   CE+EE  
keys usage_activity_by_stage create   CE+EE  
merge_requests usage_activity_by_stage create   CE+EE  
projects_with_disable_overriding_approvers_per_merge_request usage_activity_by_stage create   CE+EE  
projects_without_disable_overriding_approvers_per_merge_request usage_activity_by_stage create   CE+EE  
remote_mirrors usage_activity_by_stage create   CE+EE  
snippets usage_activity_by_stage create   CE+EE  
merge_requests_users usage_activity_by_stage_monthly create   CE+EE Unique count of users who used a merge request
projects_enforcing_code_owner_approval usage_activity_by_stage create   EE  
merge_requests_with_optional_codeowners usage_activity_by_stage create   EE  
merge_requests_with_required_codeowners usage_activity_by_stage create   EE  
projects_imported_from_github usage_activity_by_stage create   EE  
projects_with_repositories_enabled usage_activity_by_stage create   EE  
protected_branches usage_activity_by_stage create   EE  
suggestions usage_activity_by_stage create   EE  
approval_project_rules usage_activity_by_stage create   EE Number of project approval rules
approval_project_rules_with_target_branch usage_activity_by_stage create   EE Number of project approval rules with not default target branch
clusters usage_activity_by_stage monitor   CE+EE  
clusters_applications_prometheus usage_activity_by_stage monitor   CE+EE  
operations_dashboard_default_dashboard usage_activity_by_stage monitor   CE+EE  
operations_dashboard_users_with_projects_added usage_activity_by_stage monitor   EE  
projects_prometheus_active usage_activity_by_stage monitor   EE  
projects_with_error_tracking_enabled usage_activity_by_stage monitor   EE  
projects_with_tracing_enabled usage_activity_by_stage monitor   EE  
events usage_activity_by_stage manage   CE+EE  
groups usage_activity_by_stage manage   CE+EE  
users_created_at usage_activity_by_stage manage   CE+EE  
omniauth_providers usage_activity_by_stage manage   CE+EE  
ldap_keys usage_activity_by_stage manage   EE  
ldap_users usage_activity_by_stage manage   EE  
value_stream_management_customized_group_stages usage_activity_by_stage manage   EE  
projects_with_compliance_framework usage_activity_by_stage manage   EE  
ldap_servers usage_activity_by_stage manage   EE  
ldap_group_sync_enabled usage_activity_by_stage manage   EE  
ldap_admin_sync_enabled usage_activity_by_stage manage   EE  
group_saml_enabled usage_activity_by_stage manage   EE  
issues usage_activity_by_stage plan   CE+EE  
notes usage_activity_by_stage plan   CE+EE  
projects usage_activity_by_stage plan   CE+EE  
todos usage_activity_by_stage plan   CE+EE  
assignee_lists usage_activity_by_stage plan   EE  
epics usage_activity_by_stage plan   EE  
label_lists usage_activity_by_stage plan   EE  
milestone_lists usage_activity_by_stage plan   EE  
projects_jira_active usage_activity_by_stage plan   EE  
projects_jira_dvcs_server_active usage_activity_by_stage plan   EE  
projects_jira_dvcs_server_active usage_activity_by_stage plan   EE  
service_desk_enabled_projects usage_activity_by_stage plan   EE  
service_desk_issues usage_activity_by_stage plan   EE  
deployments usage_activity_by_stage release   CE+EE Total deployments
failed_deployments usage_activity_by_stage release   CE+EE Total failed deployments
projects_mirrored_with_pipelines_enabled usage_activity_by_stage release   EE Projects with repository mirroring enabled
releases usage_activity_by_stage release   CE+EE Unique release tags in project
successful_deployments usage_activity_by_stage release   CE+EE Total successful deployments
user_preferences_group_overview_security_dashboard usage_activity_by_stage secure      
ci_builds usage_activity_by_stage verify   CE+EE Unique builds in project
ci_external_pipelines usage_activity_by_stage verify   CE+EE Total pipelines in external repositories
ci_internal_pipelines usage_activity_by_stage verify   CE+EE Total pipelines in GitLab repositories
ci_pipeline_config_auto_devops usage_activity_by_stage verify   CE+EE Total pipelines from an Auto DevOps template
ci_pipeline_config_repository usage_activity_by_stage verify   CE+EE Pipelines from templates in repository
ci_pipeline_schedules usage_activity_by_stage verify   CE+EE Pipeline schedules in GitLab
ci_pipelines usage_activity_by_stage verify   CE+EE Total pipelines
ci_triggers usage_activity_by_stage verify   CE+EE Triggers enabled
clusters_applications_runner usage_activity_by_stage verify   CE+EE Unique clusters with Runner enabled
projects_reporting_ci_cd_back_to_github usage_activity_by_stage verify   EE Unique projects with a GitHub pipeline enabled
merge_requests_users usage_activity_by_stage_monthly create     Unique count of users who used a merge request
duration_s topology enablement     Time it took to collect topology data
application_requests_per_hour topology enablement     Number of requests to the web application per hour
failures topology enablement     Contains information about failed queries
nodes topology enablement     The list of server nodes on which GitLab components are running
node_memory_total_bytes topology > nodes enablement     The total available memory of this node
node_cpus topology > nodes enablement     The number of CPU cores of this node
node_uname_info topology > nodes enablement     The basic hardware architecture and OS release information on this node
node_services topology > nodes enablement     The list of GitLab services running on this node
name topology > nodes > node_services enablement     The name of the GitLab service running on this node
process_count topology > nodes > node_services enablement     The number of processes running for this service
process_memory_rss topology > nodes > node_services enablement     The average Resident Set Size of a service process
process_memory_uss topology > nodes > node_services enablement     The average Unique Set Size of a service process
process_memory_pss topology > nodes > node_services enablement     The average Proportional Set Size of a service process
server topology > nodes > node_services enablement     The type of web server used (Unicorn or Puma)

Example Usage Ping payload

The following is example content of the Usage Ping payload.

{
  "uuid": "0000000-0000-0000-0000-000000000000",
  "hostname": "example.com",
  "version": "12.10.0-pre",
  "installation_type": "omnibus-gitlab",
  "active_user_count": 999,
  "recorded_at": "2020-04-17T07:43:54.162+00:00",
  "edition": "EEU",
  "license_md5": "00000000000000000000000000000000",
  "license_id": null,
  "historical_max_users": 999,
  "licensee": {
    "Name": "ABC, Inc.",
    "Email": "email@example.com",
    "Company": "ABC, Inc."
  },
  "license_user_count": 999,
  "license_starts_at": "2020-01-01",
  "license_expires_at": "2021-01-01",
  "license_plan": "ultimate",
  "license_add_ons": {
  },
  "license_trial": false,
  "counts": {
    "assignee_lists": 999,
    "boards": 999,
    "ci_builds": 999,
    ...
  },
  "container_registry_enabled": true,
  "dependency_proxy_enabled": false,
  "gitlab_shared_runners_enabled": true,
  "gravatar_enabled": true,
  "influxdb_metrics_enabled": true,
  "ldap_enabled": false,
  "mattermost_enabled": false,
  "omniauth_enabled": true,
  "prometheus_enabled": false,
  "prometheus_metrics_enabled": false,
  "reply_by_email_enabled": "incoming+%{key}@incoming.gitlab.com",
  "signup_enabled": true,
  "web_ide_clientside_preview_enabled": true,
  "ingress_modsecurity_enabled": true,
  "projects_with_expiration_policy_disabled": 999,
  "projects_with_expiration_policy_enabled": 999,
  ...
  "elasticsearch_enabled": true,
  "license_trial_ends_on": null,
  "geo_enabled": false,
  "git": {
    "version": {
      "major": 2,
      "minor": 26,
      "patch": 1
    }
  },
  "gitaly": {
    "version": "12.10.0-rc1-93-g40980d40",
    "servers": 56,
    "clusters": 14,
    "filesystems": [
      "EXT_2_3_4"
    ]
  },
  "gitlab_pages": {
    "enabled": true,
    "version": "1.17.0"
  },
  "database": {
    "adapter": "postgresql",
    "version": "9.6.15"
  },
  "app_server": {
    "type": "console"
  },
  "avg_cycle_analytics": {
    "issue": {
      "average": 999,
      "sd": 999,
      "missing": 999
    },
    "plan": {
      "average": null,
      "sd": 999,
      "missing": 999
    },
    "code": {
      "average": null,
      "sd": 999,
      "missing": 999
    },
    "test": {
      "average": null,
      "sd": 999,
      "missing": 999
    },
    "review": {
      "average": null,
      "sd": 999,
      "missing": 999
    },
    "staging": {
      "average": null,
      "sd": 999,
      "missing": 999
    },
    "production": {
      "average": null,
      "sd": 999,
      "missing": 999
    },
    "total": 999
  },
  "analytics_unique_visits": {
    "g_analytics_contribution": 999,
    ...
  },
  "usage_activity_by_stage": {
    "configure": {
      "project_clusters_enabled": 999,
      ...
    },
    "create": {
      "merge_requests": 999,
      ...
    },
    "manage": {
      "events": 999,
      ...
    },
    "monitor": {
      "clusters": 999,
      ...
    },
    "package": {
      "projects_with_packages": 999
    },
    "plan": {
      "issues": 999,
      ...
    },
    "release": {
      "deployments": 999,
      ...
    },
    "secure": {
      "user_container_scanning_jobs": 999,
      ...
    },
    "verify": {
      "ci_builds": 999,
      ...
    }
  },
  "usage_activity_by_stage_monthly": {
    "configure": {
      "project_clusters_enabled": 999,
      ...
    },
    "create": {
      "merge_requests": 999,
      ...
    },
    "manage": {
      "events": 999,
      ...
    },
    "monitor": {
      "clusters": 999,
      ...
    },
    "package": {
      "projects_with_packages": 999
    },
    "plan": {
      "issues": 999,
      ...
    },
    "release": {
      "deployments": 999,
      ...
    },
    "secure": {
      "user_container_scanning_jobs": 999,
      ...
    },
    "verify": {
      "ci_builds": 999,
      ...
    }
  },
  "topology": {
    "duration_s": 0.013836685999194742,
    "application_requests_per_hour": 4224,
    "failures": [],
    "nodes": [
      {
        "node_memory_total_bytes": 33269903360,
        "node_cpus": 16,
        "node_uname_info": {
          "machine": "x86_64",
          "sysname": "Linux",
          "release": "4.19.76-linuxkit"
        },
        "node_services": [
          {
            "name": "web",
            "process_count": 16,
            "process_memory_pss": 233349888,
            "process_memory_rss": 788220927,
            "process_memory_uss": 195295487,
            "server": "puma"
          },
          {
            "name": "sidekiq",
            "process_count": 1,
            "process_memory_pss": 734080000,
            "process_memory_rss": 750051328,
            "process_memory_uss": 731533312
          },
          ...
        ],
        ...
      },
      ...
    ]
  }
}