Purpose/Objective(s):

Implementation of artificial intelligence (AI) and automation requires specific goals and measurable endpoints. As the use of these tools grows in radiation medicine, particularly for emerging indications like osteoarthritis (OA), we must establish evaluation criteria that match desired clinical outcomes. We developed OSTEO, a rubric for evaluation of low-dose radiation (LDRT) plans for OA, and validated its utility through comparison of automated and manual plans for knee OA.

Materials/Methods:

OSTEO defines five plan evaluation criteria for OA LDRT: Overdose (hotspot), Size (field dimensions), Target coverage, Epicenter (isocenter), and Orientation (laterality/correct joint), organized into three scored domains scored 0-3: Coverage, Hot spot, and Field Setup (Table). Field Setup integrates S, E and O as technically interdependent criteria. Evaluators used an expanded rubric with specific field size parameters and visual dose distribution examples. Automated plans were generated using CT simulation scans from previously treated manual LDRT knee OA plans. Four radiation oncologists independently evaluated plans using the OSTEO rubric. Inter-reader variability was assessed using intraclass correlation (ICC). Paired Wilcoxon rank-tests compared scores between manual and automated plans.

Results:

Four reviewers evaluated 21 automated and manual plans (168 total assessments). Inter-rater reliability for total OSTEO scores was excellent for manual (ICC 0.91) and good for automated plans (ICC 0.86). Mean total OSTEO scores were comparable between automated (range 7.9–8.3) and manual plans (range 8.0–8.5), with no significant differences in total scores across any reader.

Conclusion:

The OSTEO rubric is a simple method to standardize evaluation of LDRT plans for OA, and addresses the need for precise goal definition for successful use of AI and automation-enabled LDRT planning. OSTEO is adaptable to other joints and can be modified as planning parameters for LDRT are refined.

AI tools (Claude Opus, Anthropic) were used for language refinement. Authors are fully responsible for scientific content.

IDL = isodose line. Dmax = maximum dose.

Domain	OSTEO	3 Ideal	2 Acceptable	1 Borderline, minor revisions recommended	0 Reject, major revisions required
Hot spot	(O) Overdose	Dmax =110%	Dmax 110-115%	Dmax 115-120%	Dmax >120%
	(S) Size	See Field Setup domain below
Coverage	(T) Target	95% IDL covers entire joint	Minor cold spots in 95% IDL, 90% IDL covers joint	Minor cold spots in 90% IDL	Major cold spots in 90% IDL
Field Setup	(E) Epicenter (O) Orientation	•Field size matches desired superior/inferior, medial/lateral expansions from isocenter • Isocenter in tibiofemoral joint space • Correct joint	• Isocenter/field size variations from ideal that do not affect coverage/reproducibility • Correct joint	Deviations from isocenter/field size that do affect coverage/reproducibility	• Field not covering joint • Field overlapping with off target anatomy • Incorrect joint