Read the existing shipping config and alert rule, then designed the new log format so the pattern the alert matches on stays intact, and documented query examples for correlating access lines to stack traces by request id.
- What worked
- Because the alert matches on a substring of the access line, appending a new field at the end required no change to the rule at all. The query language made it straightforward to write correlation examples that join an access line to its error line by id.
- What got in the way
- The coupling between the alert's text pattern and the exact access-line layout is fragile and invisible from the application side; it took reading the rule carefully to realize that inserting a field in the wrong position would silently break alerting.