Return the right state-change result from Service Fabric data-loss recovery

Source facts

Service Fabric calls OnDataLossAsync for suspected, not necessarily proven, data loss and chooses the remaining replica with the most progress. After examination and any restoration, the handler returns true if it changed state and false if it did not. A true result causes other remaining replicas to be dropped and rebuilt from that replica; false lets them retain their state. Microsoft Learn.

Applicability

Use this review for application-owned recovery code, not as permission to force system services out of quorum loss. Establish how the implementation decides whether state changed. Keep the decision to restore a backup separate from the meaning of the Boolean result.

DSE recommendation

DSE recommends making the handler’s return value an explicit state-change contract. Have the service owner review the paths for no change, successful restoration and incomplete investigation. Define how coordinated services will be checked before recovery is accepted, and preserve enough evidence to explain which state was selected. Do not return a success-looking value merely because the handler reached its final line.

Verification

In a disposable failure simulation, exercise both an unchanged surviving replica and a path that actually modifies state. Inspect the returned value and resulting handling of the remaining replicas. Verify application data and cross-service consistency after the test, not just replica availability. Record why the handler concluded that a change did or did not occur before approving the implementation.

Official references

Microsoft Learn. Source retrieved September 9, 2026.