• Common developer tools
• Versioning and Release
• Changelog Entries
• Installation from Repo
• Running GitLab QA
• ChaosKube
• Developing for Kubernetes with Minikube
• Developing for Kubernetes with KinD
• Developing RSpec tests
• Naming Conventions
• Common structure for values.yaml
  • Connecting to other services
  • Sharing secrets
  • Certificates
  • Passwords
  • Auth Tokens
• Preferences on function use
  • Use nindent over indent
  • When to utilize toYaml in templates
• Developing template helpers
  • Trapping non-printed values from actions
  • Passing variables between control structures
• When to fork upstream charts
  • No changes, no fork
  • Guidelines for forking
    • Sensitive information
    • Extending functionality
• Handling configuration deprecations
• Attempt to catch problematic configurations
• Verifying registry

Development Guide

Our contribution policies can be found in CONTRIBUTING.md

Common developer tools

Developers working on charts often use the following tools:

Versioning and Release

Details on the version scheme, branching and tags can be found in release document.

Changelog Entries

All CHANGELOG.md entries should be created via the changelog entries workflow.

Installation from Repo

Details on installing from the Git repo can be found in the developer deployment documentation.

Running GitLab QA

GitLab QA can be used against a deployed cloud native GitLab installation.

Read more in the GitLab QA chart docs.

ChaosKube

Read more in the ChaosKube chart docs.

Developing for Kubernetes with Minikube

Read how to use Minikube for setting up a local Kubernetes development environment.

Developing for Kubernetes with KinD

Read how to use KinD for setting up a local Kubernetes development environment.

Developing RSpec tests

Read the notes on creating RSpec tests to validate the functionality of the chart.

Naming Conventions

We are using camelCase for our function names, and properties where they are used in values.yaml.

Example: gitlab.assembleHost

Template functions are placed into namespaces according to the chart they are associated with, and named to match the affected populated value in the target file. Note that chart global functions generally fall under the gitlab.* namespace.

Examples:

• gitlab.redis.host: provides the host name of the Redis server, as a part of the gitlab chart.
• registry.minio.url: provides the URL to the MinIO host as part of the registry chart.

Common structure for values.yaml

Many charts need to be provided with the same information, for example we need to provide the Redis and PostgreSQL connection settings to multiple charts. Here we outline our standard naming and structure for those settings.

Connecting to other services

redis:
  host: redis.example.com
  serviceName: redis
  port: 8080
    sentinels:
    - host: sentinel1.example.com
      port: 26379
  password:
    secret: gitlab-redis
    key: redis-password

• redis - the name for what the current chart needs to connect to
• host - overrides the use of serviceName, comment out by default use 0.0.0.0 as the example. If using Redis Sentinels, the host attribute needs to be set to the cluster name as specified in the sentinel.conf.
• serviceName - intended to be used by default instead of the host, connect using the Kubernetes Service name
• port - the port to connect on. Comment out by default, and use the default port as the example.
• password- defines settings for the Kubernetes Secret containing the password.
• sentinels.[].host - defines the hostname of Redis Sentinel server for a Redis HA setup.
• sentinels.[].port - defines the port on which to connect to the Redis Sentinel server. Defaults to 26379.

Note: The current Redis Sentinel support only supports Sentinels that have been deployed separately from the GitLab chart. As a result, the Redis deployment through the GitLab chart should be disabled with redis.install=false. The Secret containing the Redis password will need to be manually created before deploying the GitLab chart.

Sharing secrets

We use secrets to store sensitive information like passwords and share them among the different charts/pods.

The common fields we use them in are:

• Certificates - TLS certificates for the registry etc.
• Passwords - Sharing the Redis password.
• Auth Tokens - Sharing the inter-service auth tokens

Certificates

For example, where registry was the owning chart, and the other charts need to reference the registry certificate.

The owning chart should define its certificate secret like the following:

certificate:
  secret: <secret name>
  key: <key name inside the secret to fetch>

Other charts should share the same certificate secret like the following:

registry:
  certificate:
    secret: <secret name>
    key: <key name inside the secret to fetch>

Passwords

For example, where redis was the owning chart, and the other charts need to reference the redis password.

The owning chart should define its password secret like the following:

password:
  secret: <secret name>
  key: <key name inside the secret to fetch>

Other charts should share the same password secret like the following:

redis:
  password:
    secret: <secret name>
    key: <key name inside the secret to fetch>

Auth Tokens

The owning chart should define its authToken secret like the following:

authToken:
  secret: <secret name>
  key: <key name inside the secret to fetch>

Other charts should share the same password secret like the following:

gitaly:
  authToken:
    secret: <secret name>
    key: <key name inside the secret to fetch>

For example, where gitaly was the owning chart, and the other charts need to reference the gitaly authToken.

Preferences on function use

We have evolved a set of preferences for developing these charts, regarding the various functions available to use witin gotmpl, Sprig, and Helm. The following sections explain some of these, and reasoning behind them

Use nindent over indent

When possible, make use of the nindent function instead of the indent function. This preference is based on readability, and especially for Helm charts as complex as ours can be. The preferred use of nindent has become community wide, and is also now the default within templates generated by the helm create command.

Let’s look at two snippet examples, which easily exmplify the reasoning:

Easy to read

  gitlab.yml.erb: |
    production: &base
      gitlab:
        host: {{ template "gitlab.gitlab.hostname" . }}
        https: {{ hasPrefix "https://" (include "gitlab.gitlab.url" .) }}
        {{- with .Values.global.hosts.ssh }}
        ssh_host: {{ . | quote }}
        {{- end }}
        {{- with .Values.global.appConfig }}
        max_request_duration_seconds: {{ default (include "gitlab.appConfig.maxRequestDurationSeconds" $) .maxRequestDurationSeconds }}
        impersonation_enabled: {{ .enableImpersonation }}
        usage_ping_enabled: {{ eq .enableUsagePing true }}
        default_can_create_group: {{ eq .defaultCanCreateGroup true }}
        username_changing_enabled: {{ eq .usernameChangingEnabled true }}
        issue_closing_pattern: {{ .issueClosingPattern | quote }}
        default_theme: {{ .defaultTheme }}
        {{- include "gitlab.appConfig.defaultProjectsFeatures.configuration" $ | nindent 8 }}
        webhook_timeout: {{ .webhookTimeout }}
        {{- end }}
        trusted_proxies:
        {{- if .Values.trusted_proxies }}
          {{- toYaml .Values.trusted_proxies | nindent 10 }}
        {{- end }}
        time_zone: {{ .Values.global.time_zone | quote }}
        {{- include "gitlab.outgoing_email_settings" . | nindent 8 }}
      {{- with .Values.global.appConfig }}
      {{- if eq .incomingEmail.enabled true }}
      {{- include "gitlab.appConfig.incoming_email" . | nindent 6 }}
      {{- end }}
      {{- include "gitlab.appConfig.cronJobs" . | nindent 6 }}
      gravatar:

Hard to read

  gitlab.yml.erb: |
    production: &base
      gitlab:
        host: {{ template "gitlab.gitlab.hostname" . }}
        https: {{ hasPrefix "https://" (include "gitlab.gitlab.url" .) }}
{{- with .Values.global.hosts.ssh }}
        ssh_host: {{ . | quote }}
{{- end }}
{{- with .Values.global.appConfig }}
        max_request_duration_seconds: {{ default (include "gitlab.appConfig.maxRequestDurationSeconds" $) .maxRequestDurationSeconds }}
        impersonation_enabled: {{ .enableImpersonation }}
        usage_ping_enabled: {{ eq .enableUsagePing true }}
        default_can_create_group: {{ eq .defaultCanCreateGroup true }}
        username_changing_enabled: {{ eq .usernameChangingEnabled true }}
        issue_closing_pattern: {{ .issueClosingPattern | quote }}
        default_theme: {{ .defaultTheme }}
{{- include "gitlab.appConfig.defaultProjectsFeatures.configuration" $ | indent 8 }}
        webhook_timeout: {{ .webhookTimeout }}
{{- end }}
        trusted_proxies:
{{- if .Values.trusted_proxies }}
{{- toYaml .Values.trusted_proxies | indent 10 }}
{{- end }}
        time_zone: {{ .Values.global.time_zone | quote }}
{{- include "gitlab.outgoing_email_settings" . | indent 8 }}
{{- with .Values.global.appConfig }}
{{- if eq .incomingEmail.enabled true }}
{{- include "gitlab.appConfig.incoming_email" . | indent 6 }}
{{- end }}
{{- include "gitlab.appConfig.cronJobs" . | indent 6 }}
      gravatar:

Related issue: #729 Refactoring: Helm templates

When to utilize toYaml in templates

It is frowned upon to default to utilizing a toYaml in the template files as this will put undue burden on supporting all functionalities of both Kuberentes and desired community configurations. We primary focus on providing a reasonable default using the bare minimum configuration. Our secondary focus would be to provide the ability to override the defaults for more advanced users of Kubernetes. This should be done on a case-by-case basis as there are certainly scenarios where either option may be too cumbersome to support, or provides an unnecessarily complex template to maintain.

An good example of a reasonable default with the ability to override can be found in the Horizontal Pod Autoscaler configuration for the registry subchart. We default to providing the bare minimum that can easily be supported, by exposing a specific configuration of controlling the HPA via the CPU Utilization and exposing only one configuration option to the community, the targetAverageUtilization. Being that an HPA can provide much more flexibility, more advanced users may want to target different metrics and as such, is a perfect example of where we can utilize and if statement allowing the end user to provide a more complex HPA configuration in place.

  metrics:
  {{- if not .Values.hpa.customMetrics }}
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          targetAverageUtilization: {{ .Values.hpa.cpu.targetAverageUtilization }}
  {{- else -}}
    {{- toYaml .Values.hpa.customMetrics | nindent 4 -}}
  {{- end -}}

In the above example, the minimum configuration will be a simple change in the values.yaml to update the targetAverageUtilization.

Advanced users who have identified a better metric can override this overly simplistic HPA configuration by setting .customMetrics to an array containing precisely the Kubernetes API compatible configuration for the HPA metrics array.

It is important that we maintain ease of use for the more advanced users to minimize their own configuration files without it being cumbersome.

Developing template helpers

A charts template helpers are located in templates/_helpers.tpl. These contain the named templates used within the chart.

When using these templates, there a few things to keep in mind regarding the golang templating syntax.

Trapping non-printed values from actions

In the go templating syntax, all actions (indicated by {{ }}) are expected to print a string, with the exception of control structures (define, if, with, range) and variable assignment.

This means you will sometimes need to use variable assignment to trap output that is not meant to be printed.

For example:

{{- $details := .Values.details -}}
{{- $_ := set $details "serviceName" "example" -}}
{{ template "serviceHost" $details }}

In the above example, we want to add some additional data to a Map before passing it to a template function for output. We trapped the output of the set function by assigning it to the $_ variable. Without this assignment, the template would try to output the result of set (which returns the Map it modified) as a string.

Passing variables between control structures

The go templating syntax only gives us one way to assign variables, and that is by using shorthand assignment.

As a result you cannot reassign a variable that existed outside your control structure (if/with/range), and variables declared within your control structure are not available outside.

For example:

{{- define "exampleTemplate" -}}
{{- $someVar := "default" -}}
{{- if true -}}
{{-   $someVar := "desired" -}}
{{- end -}}
{{- $someVar -}}
{{- end -}}

In the above example, calling exampleTemplate will always return default because the variable that contained desired was only accessible within the if control structure.

To work around this issue, we either avoid the problem, or use a Dictionary to hold the values we want to change.

Example of avoiding the issue:

{{- define "exampleTemplate" -}}
{{- if true -}}
{{-   "desired" -}}
{{- else -}}
{{-   "default" -}}
{{- end -}}

Example of using a Dictionary:

{{- define "exampleTemplate" -}}
{{- $result := dict "value" "default" -}}
{{- if true -}}
{{-   $_ := set $result "value" "desired" -}}
{{- end -}}
{{- $result.value -}}
{{- end -}}

When to fork upstream charts

No changes, no fork

Let it be stated that any chart that does not require changes to function for our use should not be forked into this repository.

Guidelines for forking

Sensitive information

If a given chart expects that sensitive communication secrets will be presented from within environment, such as passwords or cryptographic keys, we prefer to use initContainers.

Extending functionality

There are some cases where it is needed to extend the functionality of a chart in such a way that an upstream may not accept.

Handling configuration deprecations

There are times in a development where changes in behavior require a functionally breaking change. We try to avoid such changes, but some items can not be handled without such a change.

To handle this, we have implemented the deprecations template. This template is designed to recognize properties that need to be replaced or relocated, and inform the user of the actions they need to take. This template will compile all messages into a list, and then cause the deployment to stop via a fail call. This provides a method to inform the user at the same time as preventing the deployment the chart in a broken or unexpected state.

See the documentation of the deprecations template for further information on the design, functionality, and how to add new deprecations.

Attempt to catch problematic configurations

Due to the complexity of these charts and their level of flexibility, there are some overlaps where it is possible to produce a configuration that would lead to an unpredictable, or entirely non-functional deployment. In an effort to prevent known problematic settings combinations, we have the following two patterns in place:

• We use schema validations for all our sub-charts to ensure the user-specified values meet expectations. See the documentation to learn more.
• We implement template logic designed to detect and warn the user that their configuration will not work. See the documentation of the checkConfig template for further information on the design and functionality, and how to add new configuration checks.

Verifying registry

In development mode, verifying Registry with Docker clients can be difficult. This is partly due to issues with certificate of the registry. You can either add the certificate or expose the registry over HTTP (see global.hosts.registry.https). Note that adding the certificate is more secure than the insecure registry solution.

Please keep in mind that Registry uses the external domain name of MinIO service (see global.hosts.minio.name). You may encounter an error when using internal domain names, e.g. with custom TLDs for development environment. The common symptom is that you can login to the Registry but you can’t push or pull images. This is generally because the Registry container(s) can not resolve the MinIO domain name and find the correct endpoint (you can see the errors in container logs).