Accuracy of orthognathic surgery using 3D computer-assisted surgical simulation

Objective: To evaluate the accuracy of maxilla and mandibular repositioning during two-jaw orthognathic surgery using computer-assisted surgical simulation (CASS). Materials and methods: Fifteen patients who underwent two-jaw orthognathic surgery using CASS (VSP ® Orthognathics by 3D Systems) were evaluated to assess the accuracy of the simulation. Translational and rotational discrepancies of the centroids of the maxilla and mandible and the translational discrepancy of the dental midline between the planned and actual outcomes were reported using the root mean square error (RMSE). The number of cases that exceeded limits set for clinical significance, the direction of the error in relation to the direction of planned movement and the differences between segmental and non-segmental procedures were evaluated as secondary outcomes. Results: The largest translational RMSE was 1.53 mm along the y-axis in the maxilla and 1.34 mm along the y-axis in the mandible. The largest rotational RMSE was 1.9° about the x-axis in the maxilla and 1.16° about the x-and y-axes in the mandible. The largest RMSE for the dental midline was 1.6 mm along the y-axis in the maxilla and 1.34 mm along the y-axis in the mandible. A tendency for insufficient advancement of the maxilla was noted. Conclusions: CASS is an efficient and accurate way to develop the surgical plan and transfer the plan to the patient intraoperatively. While CASS is accurate on a population level, there remains the potential for clinically significant errors to occur on an individual basis.


Introduction
Orthognathic surgery to reposition the jaws or dentoalveolar segments is required to treat individuals with severe orthodontic problems not amenable to correction by growth modification or camouflage or for which camouflage options will be detrimental to facial aesthetics. 1 Preparation and planning for orthognathic surgery by conventional methods relies on the use of clinical anthropometric data, lateral and frontal radiographs and photographs. 2,3 A relocation plan is then formulated on which hard and soft tissue predictions can be made. 2 Dental plaster models are constructed and the planned surgery is executed on the models, following which, dental splints are fabricated to assist in the repositioning of the hard tissue segments during the surgical procedure. [2][3][4] The introduction and increased utilisation of volumetric imaging coupled with technological advances in computing have brought about a paradigm shift in the surgical planning process. Advances in cone beam computed tomography (CBCT) and multislice computed tomographic (MSCT) imaging have reduced the cost of imaging as well as the radiation dose. 2 This has led to the introduction of computerassisted surgical simulation (CASS) for orthognathic surgery. Computer aided design/manufacturing (CAD/CAM) technology enables the fabrication of surgical splints that facilitate the transfer of the planned movements intraoperatively without the need for traditional plaster model surgery. 5 The CASS procedure and CAD/CAM splints have the potential to improve surgical outcomes by simulating the true position of the cuts and better visualisation of the patient's anatomy, the presence and correction of asymmetry, planned movements and any potential interferences, as well as reducing planning time. 2,4,6 Few studies have evaluated the accuracy of the CASS and CAD/CAM procedures thus far. Xia and colleagues, in a pilot study of five patients, found a median linear difference of 0.9 mm (max 1.99 mm) and a median angular difference of 1.7° (max 3.48°) between the actual and predicted centroids of the maxilla and mandible. 7 Hsu and colleagues evaluated 65 patients in a prospective multicentre study and reported the difference between the actual and planned maxillary dental midline in addition to the centroids of the maxilla and the mandible related to the root mean square deviation (RMSD). 8 With a largest positional RMSD of 1.0 mm and orientation difference of 1.5° in the maxilla and 1.1 mm and 1.8° in the mandible and 0.9 mm for the maxillary dental midline, it was concluded that the CASS protocol allowed for accurate transfer of the computerised surgical plan to the patient intraoperatively. 8 Bobek and colleagues evaluated an alternative work-up procedure of 25 cases utilising intraoral fiducial markers and mandible-first surgery and reported a largest positional RMSD of 2.1 mm for the maxilla, in the vertical direction, and 1.2 mm in the mandible. 9 The limits for clinical significance were set at 2.0 mm for positional differences, 4° for orientation differences and were reported as 1.0 mm for the mediolateral position of the maxillary dental midline in the existing literature. [7][8][9] Bobek and colleagues concluded that their modified CASS protocol for orthognathic surgery was accurate except for the vertical position of the maxilla, and this could be attributed to an adjustment of the plan by the surgeon, mandibular condyle seating issues, segmental maxillary surgery and the small sample size. 9 While the current literature supports the efficiency and accuracy of the CASS and CAD/CAM protocol for orthognathic surgery, it often costs the patient more and comes with an additional radiation dosage associated with volumetric imaging. Further research is required to validate the accuracy of CASS, particularly as there are many variations in the CASS protocol, and as the experience of the individual surgical/orthodontic team and the difficulty and complexity of the surgery may affect the accuracy of the procedure. 2,10,11

Sample population
Approval to conduct this research was granted by the University of Western Australia Human Research Ethics Council. Fifteen patients -11 females and four males, aged between 16.5-50.3 years, mean 31.2 years -undergoing orthognathic surgery with Virtual Surgical Planning (VSP ® ) Orthognathics by 3D Systems were selected from an initial pool of 24 consecutively treated patients by a private oral and maxillofacial surgeon and orthodontic team in Perth, Western Australia. Nine patients were excluded, as they did not have a CBCT scan following surgery.

Surgical planning procedure
All patients were planned for two-jaw orthognathic surgery and underwent CASS with VSP ® Orthognathics by 3D Systems in accordance with the Charlotte Method. 9,12 Following completion of the presurgical orthodontic phase, the patients underwent a clinical assessment with intraoral, lateral and frontal photos, and four alginate dental impressions were taken. With the patient in clinically-derived natural head position, a laser leveller was used to project a true vertical at the midline and a true horizontal onto the patient's face along which four radiopaque fiducial markers were placed. A wax bite with an embedded radiopaque fiducial marker was used to record the patient in centric relation (CR). The patient was then sent to the radiologist for CBCT (i-CAT, Imaging Sciences International, PA, USA) scans with the fiducial bite in place. Due to difficulty in patient positioning, one patient underwent a MSCT scan (Siemens SOMATOM Force, Siemens Healthineers, Erlangen, Germany). The alginate impressions were poured in diestone and CBCT scans with the models in occlusion in the fiducial bite, individually and in the final occlusion set by the orthodontist were obtained. The scans were then sent to 3D Systems where the engineer (D.B.) virtually recreated and segmented the skeletal and occlusal models and utilised the fiducial markers to register the occlusal models with the skeletal models. Subsequently, a web conference was scheduled whereby the orthodontist and surgeon confirmed the head orientation and centric relation position against the photographs derived from the clinical examination. The planned movements were subsequently executed and evaluated to identify bony interferences and to ensure the correction of asymmetry and yaw. Occlusal splints were fabricated using CAD/CAM and haptic technology and returned to the surgeon along with the surgical plan. Orthognathic surgery was performed and a postsurgical CBCT scan obtained within three months of the surgical date. The post-surgical CBCT data were sent to 3D Systems for analysis.

Evaluation
The evaluation of the position of the maxilla and mandible was similar to the methodology described previously by Hsu and colleagues. 8 A three-point coordinate system was used in each arch to determine the position of the maxilla and the mandible. The mesial buccal cusp tip of the left first molar in each arch was allocated as the origin (0,0,0). The distances in millimetres to the mesial buccal cusp tip of the right first molar and the midline of the central incisors in the mediolateral dimension (x-axis), anteroposterior dimension (y-axis) and supero-inferior dimension (z-axis) in the pre-surgical scan were provided by the 3D Systems engineer (Figures 1, 2 and 3). This was used to generate the co-ordinates of the right first molar and the midline and the calculated geometric centroid of the three points. The patient's right, superior and   anterior were taken as the positive direction for the respective axes.
Planned movements of the three points were provided by the engineer and applied to the presurgical coordinates to generate the coordinates and geometric centroid of the planned maxilla and mandibular position. The post-surgical CBCT data were superimposed on the pre-surgical data over stable unoperated portions of the scan, namely the cranial base. The actual postoperative movements of the three points in each arch were measured and provided by the engineer and the geometric centroids of the maxilla and mandible calculated.
The primary outcomes evaluated were the translational and rotational discrepancies of the maxilla and mandible and the translational discrepancy of the dental midlines. The translational discrepancies between the planned and actual outcomes of the centroids of the maxilla and mandible as well as the dental midline were reported as differences in the x-, y-and z-axes. The rotational discrepancies between the planned and actual outcomes were calculated by translational registration of the planned and actual centroids and then by measuring the rotation around the x-axis (pitch), y-axis (roll) and z-axis (yaw).
Further evaluation of the data to determine if there were differences between segmental and nonsegmental procedures, the number of cases that exceeded the limits set for clinical significance and the direction of discrepancy in relation to the initial movement were undertaken as secondary outcomes.
To assess reliability, three patients were randomly selected and the measurements repeated at least four weeks after the first set of measurements and intraclass correlation coefficients were calculated.

Statistical analyses and report
Data were analysed using the R environment for statistical computing. 13 Summary statistics including counts (N), means values, standard deviations (Std Dev), medians, minima and maxima are provided for continuous variables (Table I), whilst counts and percentages (%) are provided for categorical variables.
The absolute values of the discrepancies were used to avoid the error caused by negative and positive discrepancies cancelling each other.
The root mean squared error (RMSE) was used as the primary method of evaluating the absolute discrepancy between the planned and actual outcomes. Clinically  Table I. Summary statistics of absolute discrepancies by jaw.
(deg= degree, mm = millimeters, Std Dev = standard deviation) ACCURACY OF ORTHOGNATHIC SURGERY USING 3D COMPUTER PLANNING significant limits were set at 2 mm for translational discrepancies and 4° for rotational discrepancies of the maxilla and mandible and a more stringent 1 mm for the translational discrepancy of the dental midline, which is consistent with previous accuracy studies. 8,9 The root mean squared error (RMSE) for each axis of each measurement was calculated by firstly squaring all discrepancies, secondly by calculating the average of the squared discrepancies and, lastly, by calculating the square root of the average. Bland-Altman plots were produced to display the mean overall discrepancy and the 95% confidence intervals were taken for the translational discrepancies of the centroids and midlines.
The differences in measurement discrepancies between segmental and non-segmental procedures were investigated using independent t-tests for both the maxilla and mandible. Mean differences (MD), standard errors (SE) of the mean differences and p-values are provided. Significance was considered at the 5% level.
To assess the reliability of the measurements, intraclass correlation coefficients (ICC) were calculated (Table II) and Bland-Altman plots were generated. This analysis considers the midline, centroid and rotational, X, Y and Z discrepancies with repeated measures for the three patients. With the relatively small differences between the planned and actual measurements, the ICC was affected by measurement error and the small sample size. In the case of the ICC for the rotational discrepancy around the x-axis for the maxilla, which was rounded to 0.000, the worst measurement discrepancy between the first and second measurement on an individual was 1.64°.

Results
The largest translational RMSE reported was 1.53 mm along the y-axis in the maxilla and 1.34 mm along the y-axis in the mandible. The largest rotational RMSE reported was 1.9° about the x-axis in the maxilla and 1.16° about the x-and y-axes in the mandible (Table  III). For the dental midline position, the largest RMSE was 1.6 mm along the y-axis in the maxilla and 1.34 mm along the y-axis in the mandible, which exceeded the threshold set for clinical significance. The RMSE for the dental midline in the x-and z-axes was less than 1.0 mm and clinically insignificant.
The mean overall discrepancy and 95% confidence intervals (CI) from the Bland-Altman plots are presented in Table IV and Figure 4.
The number of times the threshold set for clinical significance was exceeded for each jaw and by each axis is presented in Table V. Care must be taken in the interpretation as some individuals had errors in more than one axis/jaw. In relation to the rotational discrepancy of the maxilla and mandible, only one case exceeded the 4° threshold for clinical significance about the x-axis in the maxilla. The translational discrepancy of the maxilla and mandible indicated that three individuals exceeded the threshold along the y-axis, one individual in both the maxilla and mandible, one individual in the maxilla only and one individual in the mandible only. The dental midlines had 13 individuals exceeding the threshold set for clinical significance involving the different combinations of axes.
The direction of planned movements and the direction of error is displayed in Tables VI-VIII. Of particular note was the tendency for insufficient advancement of the maxilla in 14 out of the 15 cases with an actual outcome posterior to the planned outcome.
Patients who underwent a segmental procedure had a significantly larger absolute midline discrepancy in   the z-axis than those who underwent a non-segmental procedure (MD = 0.57, SE0.18, p = 0.0064), although this estimated difference of 0.57 mm was not clinically significant. There were no statistically significant differences detected in the mandible between the segmental and non-segmental procedures.

Discussion
The results of the study confirm that CASS is accurate within the threshold set for clinical significance related to the overall positioning of the maxilla and mandible, with all translational RMSE within 2 mm and rotational discrepancies within 4°, which is consistent with previously-reported studies. 8,9 Hsu and colleagues set a 1 mm threshold for clinical significance of the maxillary dental midline position in the x-axis. 8 In the present study, a 1 mm threshold was applied to the upper and lower dental midlines in all three axes. The RMSE for the dental midline in the y-axis was 1.6 mm in the maxilla and 1.34 mm in the mandible, which exceeded this threshold. The RMSE was less than 1 mm in the x-axis and the z-axis and clinically insignificant. The dental midline in the y-axis essentially represents the anteroposterior position of the maxilla and mandible, measured between the central incisors. In retrospect, the 1 mm threshold is more clinically relevant in the x-and z-axes as these represent the horizontal and vertical position of the incisors, which are apparent to the patient on reflection in a mirror. The anteroposterior position is unlikely to be as critical and the current recommendation is that the 1 mm clinical limits for the dental midline be applied in the x-and z-axes, with 2 mm being the clinical limit for the y-axis (anteroposterior position) consistent with the overall position of the maxilla.
Upon comparing the differences between the segmental and non-segmental procedures and acknowledging the limitations of the small sample size, there was a statistically significant difference between the procedures in the vertical positioning of the maxilla; however, with an estimated difference of 0.57 mm, this was not clinically significant. While the present results suggest no clinically relevant difference between segmental and non-segmental procedures, further studies with larger sample sizes would need to be performed to draw definitive conclusions regarding the impact of the segmental procedure on overall surgical accuracy.
While the results support the accuracy of CASS on a population level with the RMSE within the clinical limits set, apart from the y-axis of the dental midline, it is important to note that on an individual basis, several cases exceeded the threshold for clinical significance. The translation centroids of three individuals and the rotation centroids of one individual revealed discrepancies exceeding the clinical significance threshold. Multiple individuals exceeded the threshold for the dental midline discrepancies. All the discrepancies that exceeded the threshold in the centroids and the majority of those that exceeded the threshold in the dental midline were the result of insufficient advancement of the maxilla. This is consistent with the results from the direction of error as 14 out of 15 individuals planned for maxillary advancement fell short of the target position. The insufficient advancement of the maxilla may be due to condylar seating issues as a result of reduced muscle tone under anaesthesia, the patient in a supine position and posterior pressure applied by the surgeon placing the condyles in a more posterior position. As the maxilla undergoes surgery first, this error is incorporated into the repositioning of the maxilla because the unoperated mandible is used as the initial reference. The newly repositioned maxilla is then used to reposition the lower jaw, which transfers the error to the mandible. Research should be directed at ways to reduce the number of individuals exceeding the threshold for clinical significance. New technological advances with wafer-less systems that reposition the maxilla using custom fabricated cutting guides and plates to locate the maxilla independent of the mandible may improve surgical accuracy and lead to a reduction in the number of individuals with clinically significant discrepancies. Further research is required to validate the accuracy of these new systems, particularly as they represent an increased cost to the patient and health care system.

Conclusions
1. Overall, CASS is an accurate and efficient way to determine the surgical plan and transfer the plan to the patient intraoperatively.
2. There was a tendency for insufficient advancement of the maxilla.
3. While CASS is accurate on a population level, there is still the potential for clinically significant errors to occur on an individual basis.