Clever Geek Handbook

Attraper les bogues sur le client: comment nous avons écrit notre système de collecte des erreurs client

Badoo . Badoo Bumble, - ( ), (Android iOS), UI.  Gelat. Error Tracking . 

, , , , .

:

  • , , , . 

  • , ;

  • , , .

— . , , , , .

— . Badoo : 

— -: - , ; 

— : ( , App Store Google Play). 

. - , , .

. , , QA-. 

Badoo : HockeyApp - JS-.

HockeyApp

HockeyApp , 2014 Microsoft , App Center. App Center : , , - Android- DexGuard mapping-, . ; , , - . 

Microsoft — 16 2019 , HockeyApp App Center. , DexGuard App Center 2019 , HockeyApp.

HockeyApp. Microsoft HockeyApp, , . .

JS-

JS-, , .

:

  • MySQL ( 10—20 );

  • ;

  • MySQL, Sphinx .

2017 -. — . - , - , - , .

, , . , 2014 , , . 

, .

  • .

  • . (, ) . , .

  • . , . .

  • — .

  • « ?», « ?».

  • - ( , , . .).

  • .

  • : , , , . .

  • Jira. Jira, .

  • Self-hosted. , . .

, , . 

SaaS-, : — . Bugsnag, TrackJS, Raygun, Rollbar Airbrake. , : , , HockeyApp. — .

Open-source-

open-source- . , .

Sentry , . ( 2018 ) :

  • ;

  • PostgreSQL, ;

  • , Python, — PHP Go;

  • (, Jira).

2018 Sentry, issue- , , — ClickHouse ( ). , . , , , . 

Gelato (General Error Logs And The Others), .

, , , , , , . : , , , .

.

, .

, .

, , , , . Jira .

:

, .

(1), (2) (3).

, .

, , — . , , , , - ().

: 

, :

, , . , , :

  • A/B- — , -;

  • ;

  • ( );

  • email- .

, . :

  1. .

  2. .

  3. .

:

.

, , , — API .

API:

  • ;

  • , «» ( « », «» API);

  • .

?

, EPS (errors per second), , , .

- , . . , , , EPS ( iOS Facebook SDK 10 2020 ). , , .

, , — . : , - — , . 

, ( ), , .

?

  1. , , — , Redis RabbitMQ.

  2. Apache Kafka, , (, - ). Kafka () Sentry.

  3. LSD (Live Streaming Daemon). , . Badoo , .

: « ?» ( ) « ?» ( ).

: Elasticsearch ClickHouse.

Elasticsearch

:

  • ;

  • , ;

  • ;

  • UPDATE ( , , , );

  • DELETE ( , , );

  • API, .

, , Elasticsearch :

  • , ( SQL, (X-Pack) Open Distro Amazon);

  • JVM, « », — PHP Go (, , ; 6.8 7.5, (, ));

  • ; , , (, deflate- LZ4, CPU, ).

ClickHouse

:

  • ; 

  • , ;

  • MySQL- , , Elasticsearch;

  • , Elasticsearch.

2018 ClickHouse :

  • DELETE ( , ; ClickHouse , ( — ) );

  • UPDATE : ClickHouse , — ( GitHub — 2018 , );

  • ( RegExp, (full scan), ).

, ( ()). , , Elasticsearch.

, Elasticsearch ClickHouse, . . , Elasticsearch, , — .

, .

:

  • -;

  • .

( ) — . , warm- ( hot-warm-cold-).

JS-, - , , ( , , , ). , , attributes . 

Sentry . Sentry , tags , , extra — .

, , — . , , , , - . — .

.

- Android-

, Android- , :

  • (code shrinking — );

  • , (optimization — );

  • , , , - (obfuscation — ).

.

:

  • ProGuard ( ); 

  • DexGuard ProGuard ( );

  • R8 Google.

, - :

o.imc: Error loading resources: Security check required
  at o.mef.b(:77)
  at o.mef.e(:23)
  at o.mef$a.d(:61)
  at o.mef$a.invoke(:23)
  at o.jij$c.a(:42)
  at o.jij$c.apply(Unknown Source:0)
  at o.wgv$c.a_(:81)
  at o.whb$e.a_(:64)
  at o.wgs$b$a.a_(:111)
  at o.wgy$b.run(:81)
  at o.vxu$e.run(:109)
  at android.os.Handler.handleCallback(Handler.java:790)
  at android.os.Handler.dispatchMessage(Handler.java:99)
  at android.os.Looper.loop(Looper.java:164)
  at android.app.ActivityThread.main(ActivityThread.java:6626)
  at java.lang.reflect.Method.invoke(Native Method)
  at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:438)
  at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:811)

, . -, . , mapping.txt, . :

AllGoalsDialogFragment -> o.a:
    java.util.LinkedHashMap goals -> c
    kotlin.jvm.functions.Function1 onGoalSelected -> e
    java.lang.String selectedId -> d
    AllGoalsDialogFragment$Companion Companion -> a
    54:73:android.view.View onCreateView(android.view.LayoutInflater,android.view.ViewGroup,android.os.Bundle) -> onCreateView
    76:76:int getTheme() -> getTheme
    79:85:android.app.Dialog onCreateDialog(android.os.Bundle) -> onCreateDialog
    93:97:void onDestroyView() -> onDestroyView

, , mapping- — -.

(, , ), , , ProGuard ( DexGuard ) ReTrace, .

Android- Kotlin, : 

  • - ;

  • mapping- Ceph ( TeamCity);

  • -.

- iOS-

- iOS- , - ( - ). : - , .

Thread 0:
0   libsystem_kernel.dylib              0x00000001bf3468b8 0x1bf321000 + 153784
1   libobjc.A.dylib                     0x00000001bf289de0 0x1bf270000 + 105952
2   Badoo                               0x0000000105c9c6f4 0x1047ec000 + 21694196
3   Badoo                               0x000000010657660c 0x1047ec000 + 30975500
4   Badoo                               0x0000000106524e04 0x1047ec000 + 30641668
5   Badoo                               0x000000010652b0f8 0x1047ec000 + 30667000
6   Badoo                               0x0000000105dce27c 0x1047ec000 + 22946428
7   Badoo                               0x0000000105dce3b4 0x1047ec000 + 22946740
8   Badoo                               0x0000000104d41340 0x1047ec000 + 5591872

. -, - (dSYM), , , .

?

  • , , , macOS. , , Linux.

  • Symbolicator Sentry, ( , ). - , as is : , Rust . 

  • Symbolic Sentry, , Rust, C-ABI — FFI.

Golang , Symbolic cgo.

, , — . , .

, , , , - . (fingerprint), . -. , , ( ).

:

  • JS- message, type origin;

  • Android - - ( );

  • iOS - (, crashed -).

, .

, Sentry, . 

, , . 

, :



More articles:


All Articles