Clever Geek Handbook

Comment restaurer Sentry après l'échec d'une mise à jour

Bonjour. Je veux parler de la façon dont la restauration de Sentry s'est déroulée après un échec de mise à jour.



Qu'est-ce que Sentry ?



Il s'agit d'un système de suivi des bogues complet open source qui prend en charge un large éventail de langages et d'environnements côté serveur, navigateur, bureau et mobiles natifs, y compris PHP, Node. js, Python, Ruby, C #, Java, Go, React, Angular, Vue, JavaScript et plus encore.



Un peu sur la façon dont tout s'est cassé simple et sans contrainte.



Nous avons vécu sur la version 9.0.0 et il est temps de mettre à jour. Ayant senti l'interface Web 10.0.0, j'ai décidé de la mettre à niveau. C'était la plus grosse erreur!



La mise à jour a eu lieu comme d'habitude. Commencez par passer à 9.1.2, puis à 10.0.0 (sinon c'est impossible). Ensuite, j'ai découvert que le travailleur ne démarre pas. La raison en est l'ajout de nouvelles applications et dépendances.



Les tentatives de création d'applications et de les pousser dans k8 ont échoué.



Il a été décidé de revenir en arrière. Mais ici, j'ai été déçu. Après la migration vers la version 10, certaines des données sont déplacées vers Clickhouse et supprimées de Postgres.



Après 3 heures de travail et l'implication d'un expert DBA, les données ont été remises en état de fonctionnement.



Comment?



Nous avons créé une instance PG à côté, supprimé le schéma de la version 9.0.0 et lancé une récupération tabulaire. Bien sûr, il y avait des erreurs et un fichier.



L'histoire ne s'arrête pas là. Il est temps de passer à la version 9.1.2.



Recueilli l'image, exécutez: 



sentry upgrade


et nous voyons ce merveilleux message: 



 ! I'm not trusting myself; either fix this yourself by fiddling
 ! with the south_migrationhistory table, or pass --delete-ghost-migrations
 ! to South to have it delete ALL of these records (this may not be good).
Exception in thread Thread-1 (most likely raised during interpreter shutdown)


Après une longue recherche, essais et erreurs, une solution a été trouvée.



Vous devez restaurer le schéma de la version actuelle.



Pour ce faire, nous clonons le référentiel: 



git clone https://github.com/getsentry/onpremise.git


Nous allons à la branche 9.1.2, puisque la branche 9.0.0 n'est plus là. 



cd onpremise && git checkout tags/9.1.2


Ensuite, nous éditons l'image dans le Dockerfile: 



#cat Dockerfile 
ARG SENTRY_IMAGE
FROM ${SENTRY_IMAGE:-sentry:9.0.0}-onbuild
Puis éditez le fichier de composition



docker-compose.yml 
# NOTE: This docker-compose.yml is meant to be just an example of how
# you could accomplish this on your own. It is not intended to work in
# all use-cases and must be adapted to fit your needs. This is merely
# a guideline.

# See docs.getsentry.com/on-premise/server/ for full
# instructions

version: '3.4'

x-defaults: &defaults
  restart: unless-stopped
  build:
    context: .
  depends_on:
    - redis
   # - postgres
    - memcached
    - smtp
  env_file: .env
  environment:
    SENTRY_MEMCACHED_HOST: memcached
    SENTRY_REDIS_HOST: redis
    SENTRY_POSTGRES_HOST: 'sentry.cl.ats'
    SENTRY_DB_USER: 'sentryDB'
    SENTRY_DB_NAME: 'sentry_user'
    SENTRY_DB_PASSWORD: 'sentry_passwd'
    SENTRY_POSTGRES_PORT: 5432
    SENTRY_EMAIL_HOST: smtp
  volumes:
    - sentry-data:/var/lib/sentry/files


services:
  smtp:
    restart: unless-stopped
    image: tianon/exim4

  memcached:
    restart: unless-stopped
    image: memcached:1.5-alpine

  redis:
    restart: unless-stopped
    image: redis:3.2-alpine

  #postgres:
  #  restart: unless-stopped
  #  image: postgres:9.5
  #  environment:
  #    POSTGRES_HOST_AUTH_METHOD: 'trust'
  #  volumes:
  #    - sentry-postgres:/var/lib/postgresql/data

  web:
    <<: *defaults
    ports:
      - '9000:9000'

  cron:
    <<: *defaults
    command: run cron

  worker:
    <<: *defaults
    command: run worker


volumes:
    sentry-data:
      external: true
    #sentry-postgres:
    #  external: true




Peu d'explications. Puisque j'ai une instance de base de données distincte, je saute la création via compose, et le volume sentry-postgres n'est pas non plus nécessaire.



Après ces manipulations, on démarre l'assemblage: 



./install.sh


Dans le processus, une erreur peut apparaître, que j'ai mentionnée ci-dessus. Nous ne faisons pas attention et une fois le processus terminé, exécutez la commande suivante: 



docker-compose run --rm web sentry django migrate --delete-ghost-migrations


Conclusion approximative 
Creating network "onpremise_default" with the default driver
Creating onpremise_memcached_1 ... done
Creating onpremise_smtp_1      ... done
Creating onpremise_redis_1     ... done
21:30:26 [INFO] sentry.plugins.github: apps-not-configured
Running migrations for sentry:
- Nothing to migrate.
 - Loading initial data for sentry.
Installed 0 object(s) from 0 fixture(s)
Running migrations for nodestore:
- Nothing to migrate.
 - Loading initial data for nodestore.
Installed 0 object(s) from 0 fixture(s)
Running migrations for search:
- Nothing to migrate.
 - Loading initial data for search.
Installed 0 object(s) from 0 fixture(s)
Running migrations for social_auth:
- Nothing to migrate.
 - Loading initial data for social_auth.
Installed 0 object(s) from 0 fixture(s)
Running migrations for tagstore:
- Nothing to migrate.
 - Loading initial data for tagstore.
Installed 0 object(s) from 0 fixture(s)
Running migrations for jira_ac:
- Nothing to migrate.
 - Loading initial data for jira_ac.
Installed 0 object(s) from 0 fixture(s)
Running migrations for hipchat_ac:
- Nothing to migrate.
 - Loading initial data for hipchat_ac.
Installed 0 object(s) from 0 fixture(s)




Ensuite, vous pouvez exécuter la mise à niveau. 



docker-compose run --rm web sentry upgrade


Sortie standard 
Starting onpremise_smtp_1      ... done
Starting onpremise_memcached_1 ... done
Starting onpremise_redis_1     ... done
21:30:48 [INFO] sentry.plugins.github: apps-not-configured
Syncing...
Creating tables ...
Installing custom SQL ...
Installing indexes ...
Installed 0 object(s) from 0 fixture(s)

Synced:
 > django.contrib.admin
 > django.contrib.auth
 > django.contrib.contenttypes
 > django.contrib.messages
 > django.contrib.sessions
 > django.contrib.sites
 > django.contrib.staticfiles
 > crispy_forms
 > debug_toolbar
 > raven.contrib.django.raven_compat
 > rest_framework
 > sentry.plugins.sentry_interface_types
 > sentry.plugins.sentry_mail
 > sentry.plugins.sentry_urls
 > sentry.plugins.sentry_useragents
 > sentry.plugins.sentry_webhooks
 > sudo
 > south
 > sentry_plugins.slack

Not synced (use migrations):
 - sentry
 - sentry.nodestore
 - sentry.search
 - social_auth
 - sentry.tagstore
 - sentry_plugins.jira_ac
 - sentry_plugins.hipchat_ac
(use ./manage.py migrate to migrate these)
Running migrations for sentry:
- Nothing to migrate.
 - Loading initial data for sentry.
Installed 0 object(s) from 0 fixture(s)
Running migrations for nodestore:
- Nothing to migrate.
 - Loading initial data for nodestore.
Installed 0 object(s) from 0 fixture(s)
Running migrations for search:
- Nothing to migrate.
 - Loading initial data for search.
Installed 0 object(s) from 0 fixture(s)
Running migrations for social_auth:
- Nothing to migrate.
 - Loading initial data for social_auth.
Installed 0 object(s) from 0 fixture(s)
Running migrations for tagstore:
- Nothing to migrate.
 - Loading initial data for tagstore.
Installed 0 object(s) from 0 fixture(s)
Running migrations for jira_ac:
- Nothing to migrate.
 - Loading initial data for jira_ac.
Installed 0 object(s) from 0 fixture(s)
Running migrations for hipchat_ac:
- Nothing to migrate.
 - Loading initial data for hipchat_ac.
Installed 0 object(s) from 0 fixture(s)
Creating missing DSNs
Correcting Group.num_comments counter




Ainsi, le circuit est rétabli. Vous pouvez passer à la version 9.1.2.



Pour ce faire, dans le Dockerfile, changez la version en 9.1.2: 



ARG SENTRY_IMAGE
FROM ${SENTRY_IMAGE:-sentry:9.1.2}-onbuild
Et exécutez install.sh 



./install.sh


Si tout est fait correctement, le processus se terminera sans erreurs. La sortie devrait être quelque chose comme ceci:



Sortie standard 
# ./install.sh 
Checking minimum requirements...

Creating volumes for persistent storage...
Created sentry-data.

.env already exists, skipped creation.

Building and tagging Docker images...

smtp uses an image, skipping
memcached uses an image, skipping
redis uses an image, skipping
Building web
Step 1/2 : ARG SENTRY_IMAGE
Step 2/2 : FROM ${SENTRY_IMAGE:-sentry:9.1.2}-onbuild
# Executing 4 build triggers
 ---> Running in 6c97f9fcaf63
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Removing intermediate container 6c97f9fcaf63
 ---> Running in 9e0f65ee3af6
Removing intermediate container 9e0f65ee3af6
 ---> Running in 09754c44319c
Removing intermediate container 09754c44319c
 ---> a12fa31c2675

Successfully built a12fa31c2675
Successfully tagged onpremise_web:latest
Building cron
Step 1/2 : ARG SENTRY_IMAGE
Step 2/2 : FROM ${SENTRY_IMAGE:-sentry:9.1.2}-onbuild
# Executing 4 build triggers
 ---> Using cache
 ---> Using cache
 ---> Using cache
 ---> Using cache
 ---> a12fa31c2675

Successfully built a12fa31c2675
Successfully tagged onpremise_cron:latest
Building worker
Step 1/2 : ARG SENTRY_IMAGE
Step 2/2 : FROM ${SENTRY_IMAGE:-sentry:9.1.2}-onbuild
# Executing 4 build triggers
 ---> Using cache
 ---> Using cache
 ---> Using cache
 ---> Using cache
 ---> a12fa31c2675

Successfully built a12fa31c2675
Successfully tagged onpremise_worker:latest

Docker images built.

Generating secret key...
Secret key written to .env

Setting up database...
Starting onpremise_smtp_1      ... done
Starting onpremise_redis_1     ... done
Starting onpremise_memcached_1 ... done
21:35:07 [WARNING] sentry.utils.geo: settings.GEOIP_PATH_MMDB not configured.
21:35:10 [INFO] sentry.plugins.github: apps-not-configured
Syncing...
Creating tables ...
Installing custom SQL ...
Installing indexes ...
Installed 0 object(s) from 0 fixture(s)
Migrating...
Running migrations for sentry:
 - Migrating forwards to 0472_auto__add_field_sentryapp_author.
 > sentry:0424_auto__add_field_integration_status
 > sentry:0425_auto__add_index_pullrequest_organization_id_merge_commit_sha
 > sentry:0425_remove_invalid_github_idps
 > sentry:0426_auto__add_sentryappinstallation__add_sentryapp__add_field_user_is_sent
 > sentry:0427_auto__add_eventattachment__add_unique_eventattachment_project_id_event
 > sentry:0428_auto__add_index_eventattachment_project_id_date_added
 > sentry:0429_auto__add_integrationexternalproject__add_unique_integrationexternalpr
 > sentry:0430_auto__add_field_organizationintegration_status
 > sentry:0431_auto__add_field_externalissue_metadata
 > sentry:0432_auto__add_field_relay_is_internal
 > sentry:0432_auto__add_index_userreport_date_added__add_index_eventattachment_date_
 > sentry:0433_auto__add_field_relay_is_internal__add_field_userip_country_code__add_
 > sentry:0434_auto__add_discoversavedqueryproject__add_unique_discoversavedqueryproj
 > sentry:0435_auto__add_field_discoversavedquery_created_by
 > sentry:0436_rename_projectdsymfile_to_projectdebugfile
 > sentry:0437_auto__add_field_sentryapp_status
 > sentry:0438_auto__add_index_sentryapp_status__chg_field_sentryapp_proxy_user__chg_
 > sentry:0439_auto__chg_field_sentryapp_owner
 > sentry:0440_auto__del_unique_projectdebugfile_project_debug_id__add_index_projectd
 > sentry:0441_auto__add_field_projectdebugfile_data
 > sentry:0442_auto__add_projectcficachefile__add_unique_projectcficachefile_project_
 > sentry:0443_auto__add_field_organizationmember_token_expires_at
 > sentry:0443_auto__del_dsymapp__del_unique_dsymapp_project_platform_app_id__del_ver
 > sentry:0444_auto__add_sentryappavatar__add_field_sentryapp_redirect_url__add_field
 > sentry:0445_auto__add_promptsactivity__add_unique_promptsactivity_user_feature_org
 > sentry:0446_auto__add_index_promptsactivity_project_id
 > sentry:0447_auto__del_field_promptsactivity_organization__add_field_promptsactivit
 > sentry:0448_auto__add_field_sentryapp_is_alertable
 > sentry:0449_auto__chg_field_release_owner
 > sentry:0450_auto__del_grouphashtombstone__del_unique_grouphashtombstone_project_ha
 > sentry:0451_auto__del_field_projectbookmark_project_id__add_field_projectbookmark_
 > sentry:0452_auto__add_field_sentryapp_events
 > sentry:0452_auto__del_field_releaseenvironment_organization_id__del_field_releasee
 > sentry:0453_auto__add_index_releasefile_release_name
 > sentry:0454_resolve_duplicate_0452
 > sentry:0455_auto__add_field_groupenvironment_first_seen
 > sentry:0456_auto__add_dashboard__add_unique_dashboard_organization_title__add_widg
 > sentry:0457_auto__add_field_savedsearch_is_global__chg_field_savedsearch_project__
 > sentry:0457_auto__add_monitorcheckin__add_monitor__add_index_monitor_type_next_che
 > sentry:0458_global_searches_data_migration
Saved Searchs: 100% |########################################################################################################################################################################| Time: 0:00:07
 > sentry:0459_global_searches_unique_constraint
 > sentry:0460_auto__add_field_servicehook_organization_id
 > sentry:0461_event_attachment_indexes
 > sentry:0462_auto__add_servicehookproject
 > sentry:0462_releaseenvironment_project_id
 > sentry:0463_backfill_service_hook_project
 > sentry:0464_auto__add_sentryappcomponent__add_field_sentryapp_schema
 > sentry:0464_groupenvironment_foreignkeys
 > sentry:0465_sync
 > sentry:0466_auto__add_platformexternalissue__add_unique_platformexternalissue_grou
 > sentry:0467_backfill_integration_status
 > sentry:0468_auto__add_field_projectdebugfile_code_id__add_index_projectdebugfile_p
 > sentry:0468_recent_search
 > sentry:0469_fix_state
 > sentry:0470_org_saved_search
 > sentry:0471_global_saved_search_types
 > sentry:0472_auto__add_field_sentryapp_author
The following content types are stale and need to be deleted:

    sentry | dsymobject
    sentry | dsymapp
    sentry | useridentity
    sentry | dsymsymbol
    sentry | dsymsdk
    sentry | globaldsymfile
    sentry | versiondsymfile
    sentry | projectdsymfile
    sentry | grouphashtombstone
    sentry | minidumpfile
    sentry | dsymbundle

Any objects related to these content types by a foreign key will also
be deleted. Are you sure you want to delete these content types?
If you're unsure, answer 'no'.

    Type 'yes' to continue, or 'no' to cancel: yes
Running migrations for sentry.nodestore:
- Nothing to migrate.
Running migrations for sentry.search:
- Nothing to migrate.
Running migrations for social_auth:
- Nothing to migrate.
Running migrations for sentry.tagstore:
- Nothing to migrate.
Running migrations for sentry_plugins.hipchat_ac:
- Nothing to migrate.
Running migrations for sentry_plugins.jira_ac:
- Nothing to migrate.

Synced:
 > django.contrib.admin
 > django.contrib.auth
 > django.contrib.contenttypes
 > django.contrib.messages
 > django.contrib.sessions
 > django.contrib.sites
 > django.contrib.staticfiles
 > crispy_forms
 > debug_toolbar
 > rest_framework
 > sentry.plugins.sentry_interface_types
 > sentry.plugins.sentry_mail
 > sentry.plugins.sentry_urls
 > sentry.plugins.sentry_useragents
 > sentry.plugins.sentry_webhooks
 > sudo
 > south

Migrated:
 - sentry
 - sentry.nodestore
 - sentry.search
 - social_auth
 - sentry.tagstore
 - sentry_plugins.hipchat_ac
 - sentry_plugins.jira_ac
Creating missing DSNs
Correcting Group.num_comments counter
Cleaning up...

----------------
You're all done! Run the following command get Sentry running:

  docker-compose up -d




Voilà, la version 9.1.2 est installée.



Conclusions.



Ce sont des vérités communes connues de tous. Prenez votre temps, faites des sauvegardes et vérifiez tout plusieurs fois.



Merci de votre attention!


More articles:


All Articles