1. Home
  2. To appear
  3. Up-and-Down: The Most Popular, Most Reli ...

The New England Journal of Statistics in Data Science


Up-and-Down: The Most Popular, Most Reliable, and Most Overlooked Dose-Finding Design
Pub. online: 17 December 2024      Type: Methodology Article      Open accessOpen Access
Area: Statistical Methodology
Accepted
5 November 2024
Published
17 December 2024

Abstract

Up-and-Down designs (UDDs) are ubiquitous for dose-finding in a wide variety of scientific, engineering, and clinical fields. They are defined by a few simple rules that generate a random walk around the target percentile. UDDs’ combination of robust, tractable behavior, straightforward usage, and good dose-finding performance, has won the trust of practitioners and their consulting analysts across fields and continents. In contrast, in recent decades the statistical dose-finding design field has turned a cold shoulder towards UDDs, and it is quite possible that many younger dose-finding methods researchers are not even aware of this design approach.
We present a concise overview of UDDs and their current state-of-the-art methodology, with references for further inquiry. We also revisit the performance comparison between UDDs and novel, more complicated design approaches such as the Continual Reassessment Method and the Bayesian Optimal Interval design, which we group under the term “Aim-for-Target” designs. UDDs fare very well in the comparison, particularly in terms of robustness to sources of variability.

References

[1] 
Anscombe, F. J. (1956). On Estimating Binomial Response Relations. Biometrika 43 461–464. https://doi.org/10.1093/biomet/43.3-4.461. MR0081598
[2] 
ASTM (1991). Standard test method for estimating acute oral toxicity in rats. American Society for Testing and Materials. 1163–90.
[3] 
Babb, J., Rogatko, A., Rogatko, A. and Zacks, S. (1998). Cancer Phase I Clinical Trials: Efficient Dose Escalation with Overdose Control. Stat. Med. 17 1103–1120.
[4] 
Baldi Antognini, A., Bortot, P. and Giovagnoli, A. (2008). Randomized group up and down experiments. Annals of the Institute of Statistical Mathematics 60 45–59. https://doi.org/10.1007/s10463-006-0081-5. MR2400060
[5] 
Brownlee, K. A., Hodges Jr., J. L. and Rosenblatt, M. (1953). The up-and-down method with small samples. JASA 48 262–277. MR0055644
[6] 
Carter, S. K. (1973). Study design principles in the clinical evaluation of new drugs as developed by the chemotherapy programme of the National Cancer Institute. In The Design of Clinical Trials in Cancer Therapy (M. J. Staquet, ed.) 242–289. Editions Scientific Europe, Brussels.
[7] 
Cormen, T. H., Leiserson, C. E., Rivest, R. L. and Stein, C. (2022) Introduction to algorithms, 4th Edition. MIT press. MR2572804
[8] 
Derman, C. (1957). Non-parametric up-and-down experimentation. Ann. Math. Stat. 28 795–798. https://doi.org/10.1214/aoms/1177706895. MR0090956
[9] 
Diaconis, P. and Stroock, D. (1991). Geometric Bounds for Eigenvalues of Markov Chains. Ann. App. Prob. 1 36–61. MR1097463
[10] 
Dixon, W. J. and Mood, A. (1948). A method for obtaining and analyzing sensitivity data. JASA 13 109–126.
[11] 
DOD (2001). MIL-STD-1751A – Safety and Performance Tests for the Qualification of Explosives (high explosives, propellants, and pyrotechnics). United States Department of Defense.
[12] 
Durham, S. D. and Flournoy, N. (1995). Up-and-down Designs I: Stationary Treatment Distributions. In Adaptive Designs (N. Flournoy and W. F. Rosenberger, eds.) 139–157. Institute of Mathematical Statistics. https://doi.org/10.1214/lnms/1215451483. MR1477678
[13] 
Durham, S. D., Flournoy, N. and Rosenberger, W. F. (1997). A Random Walk Rule for Phase I Clinical Trials. Biometrics 53 745–760.
[14] 
Durham, S. D. and Flournoy, N. (1994). Random Walks for Quantile Estimation. In Statistical Decision Theory and Related Topics, V (S. S. Gupta and J. O. Berger, eds.) 467–476. Springer-Verlag Inc. MR1286322
[15] 
Firth, D. (1993). Bias reduction of maximum likelihood estimates. Biometrika 80(1) 27–38. https://doi.org/10.1093/biomet/80.1.27. MR1225212
[16] 
Flournoy, N. and Oron, A. P. (2015). Up-and-down designs for dose-finding. In Handbook of Design and Analysis of Experiments (D. Bingham, A. M. Dean, M. Morris and J. Stufken, eds.) 24, 862–898. CRC Press, Chapman Hall. MR3699370
[17] 
Flournoy, N. and Oron, A. P. (2020). Bias Induced by Adaptive dose-finding designs. Journal of Applied Statistics 47(13-15) 2431–2442. https://doi.org/10.1080/02664763.2019.1649375. https://doi.org/10.1080/02664763.2019.1649375. MR4149564
[18] 
Flournoy, N., Durham, S. D. and Rosenberger, W. F. (1995). Toxicity in Sequential Dose-response Experiments. Sequential Analysis 14 217–227. https://doi.org/10.1080/07474949508836333. MR1365660
[19] 
Flournoy, N., Moler, J. and Plo, F. (2020). Performance measures in dose-finding experiments. International Statistical Review 88(3) 728–751. https://doi.org/10.1111/insr.12363. MR4180676
[20] 
Garcìa-Perez, M. A. (1998). Forced-Choice staircases with fixed step sizes: asymptotic and small-sample properties. Vision Res. 38 1861–1881.
[21] 
George, R. B., McKeen, D., Columb, M. O. and Habib, A. S. (2010). Up-down determination of the 90% effective dose of phenylephrine for the treatment of spinal anesthesia-induced hypotension in parturients undergoing cesarean delivery. Anesthesia & Analgesia 110(1) 154–158.
[22] 
Gezmu, M. The geometric up-and-down design for allocating dosage levels (1996). PhD thesis, American University, Washington, DC. MR2695534
[23] 
Gezmu, M. and Flournoy, N. (2006). Group up-and-down designs for dose-finding. J Stat. Plan. Inf. 136(6) 1749–1764. https://doi.org/10.1016/j.jspi.2005.08.002. MR2255594
[24] 
Giovagnoli, A. and Pintacuda, N. (1998). Properties of Frequency Distributions Induced by General ‘up-and-down’ Methods for Estimating Quantiles. J Stat. Plan. Inf. 74 51–63. https://doi.org/10.1016/S0378-3758(98)00076-7. MR1665120
[25] 
Gorla, C., Rosa, F., Conrado, E. and Concli, F. (2017). Bending Fatigue Strength of Case Carburized and Nitrided Gear Steels for Aeronautical Applications. International Journal of Applied Engineering Research 12.
[26] 
Heijmans, R. (1999). When does the expectation of a ratio equal the ratio of expectations? Statistical Papers 40 107–115. https://doi.org/10.1007/BF02927114. MR1668879
[27] 
Hughes, B. D. (1995) Random Walks and Random Environments. Vol. 1. Oxford Science Publications. The Clarendon Press Oxford University Press, New York. Random walks. MR1420619
[28] 
Iasonos, A. and O’Quigley, J. (2014). Adaptive dose-finding studies: a review of model-guided phase I clinical trials. Journal of Clinical Oncology 32(23) 2505–2511.
[29] 
ISO (2012). International Organization of Standardization. 12107 Metallic materials–Fatigue testing–Statistical planning and analysis of data. Geneva.
[30] 
ISO (2016). International Organization of Standardization. 14801 Dentistry–Implants–Dynamic loading test for endosseous dental implants. Geneva.
[31] 
Ivanova, A., Flournoy, N. and Chung, Y. (2007). Cumulative cohort design for dose-finding. J Stat. Plan. Inf. 137 2316–2317. https://doi.org/10.1016/j.jspi.2006.07.009. MR2325437
[32] 
Ivanova, A., Montazer-Haghighi, A., Mohanty, S. G. and Durham, S. D. (2003). Improved Up-and-down Designs for Phase I Trials. Stat. Med. 22(1) 69–82.
[33] 
Ji, Y., Liu, P., Li, Y. and Nebiyou Bekele, B. (2010). A modified toxicity probability interval method for dose-finding trials. Clinical Trials 7(6) 653–663.
[34] 
JSME (1981). Standard method of statistical fatigue testing. Japan Society of Mechanical Engineers, Japan. JSME S 002.
[35] 
Langlie, H. J. (1962). A Reliability Test Method for “One-Shot” Items. Technical Report No. U-1792, Ford Motor Company, Ford Motor Company Aeronautics Division. https://apps.dtic.mil/sti/citations/tr/ADP014612.
[36] 
Lee, S. M. and Cheung, Y. K. (2009). Model Calibration in the continual reassessment method. Clinical Trials 6 227–238.
[37] 
Lee, S. M. and Cheung, Y. K. (2011). Calibration of prior variance in the Bayesian continual reassessment method. Stat. Med. 30 2081–2089. https://doi.org/10.1002/sim.4139. MR2829158
[38] 
Liu, S. and Yuan, Y. (2015). Bayesian optimal interval designs for phase I clinical trials. Journal of the Royal Statistical Society: Series C: Applied Statistics 507–523. https://doi.org/10.1111/rssc.12089. MR3325461
[39] 
Maeda, A., Villela-Franyutti, D., Lumbreras-Marquez, M. I., Murthy, A., Fields, K. G., Justice, S. and Tsen, L. C. (2023). Labor analgesia initiation with Dural puncture Epidural Versus Conventional Epidural techniques: a Randomized biased-Coin Sequential Allocation Trial to determine the effective dose for 90% of patients of Bupivacaine. Anesthesia & Analgesia 10–1213.
[40] 
Morris, M. D. (1988). Small-Sample Confidence Limits for Parameters under Inequality Constraints with Application to Quantal Bioassay. Biometrics 44 1083–1092. https://doi.org/10.2307/2531737. MR0981001
[41] 
Narayana, T. V. Sequential procedures in probit analysis (1953). PhD thesis, University of North Carolina. MR2938682
[42] 
NATO (1999) STANAG 4489 – Explosives, impact sensititity test. North Atlantic Treaty Organization.
[43] 
NIEHS (2001). The revised up-and-down procedure: A Test method for Determining the Acute Oral Toxicity of Chemicals. Technical Report No. 2-4501, Washington D.C.
[44] 
Novik, G. P. and Christensen, D. (2024). Increased impact sensitivity in ageing high explosives; analysis of Amatol extracted from explosive remnants of war. Royal Society open science 11(3) 231344.
[45] 
OECD (2022) Test No. 425: Acute Oral Toxicity: Up-and-Down Procedure. https://www.oecd-ilibrary.org/content/publication/9789264071049-en.
[46] 
O’Quigley, J., Pepe, M. and Fisher, L. (1990). Continual reassessment method: a practical design for phase 1 clinical trials in cancer. Biometrics 33–48. https://doi.org/10.2307/2531628. MR1059105
[47] 
Oron, A. P. (2017). Up-and-Down Designs Enhanced with SPRT Rules for Phase I Cancer Trials. In Society for Clinical Trials Annual Meeting, Liverpool. SCT.
[48] 
Oron, A. P. and Hoff, P. D. (2013). Small–sample behavior of novel Phase I cancer trial designs. Clinical Trials 10(1) 63–80.
[49] 
Oron, A. P. and Flournoy, N. (2017). Centered isotonic regression: point and interval estimation for dose-response studies. Journal of Biopharmaceutical Statistics.
[50] 
Oron, A. P. and Hoff, P. D. (2009). The k-in-a-row up-and-down design, revisited. Stat. Med. 28(13) 1805–1820. https://doi.org/10.1002/sim.3590. https://doi.org/10.1002/sim.3590. MR2751599
[51] 
Oron, A. P., Azriel, D. and Hoff, P. D. (2011). Dose–finding designs: The role of convergence properties. Int. J Biostat. 7(1) 39. https://doi.org/10.2202/1557-4679.1298. MR2873999
[52] 
Oron, A. P., Souter, M. J. and Flournoy, N. (2022). Understanding research methods: Up-and-down designs for dose-finding. Anesthesiology 137(2) 137–150.
[53] 
Pace, N. L. and Stylianou, M. P. (2007). Advances in and Limitations of Up-and-down Methodology: A Précis of Clinical Use, Study Design, and Dose Estimation in Anesthesia Research. Anesthesiology 107(1) 144–152.
[54] 
Parasuraman, S. (2011). Toxicological screening. Journal of Pharmacology and Pharmacotherapeutics 2(2) 74–79. https://doi.org/10.4103/0976-500X.81895.
[55] 
Paul, R. K., Rosenberger, W. F. and Flournoy, N. (2004). Quantile estimation following non-parametric phase I clinical trials with ordinal response. Statistics in medicine 23(16) 2483–2495.
[56] 
Robbins, H. and Monro, S. (1951). A stochastic approximation method. Ann. Math. Statistics 22 400–407. MR0042668 (13,144j).
[57] 
Shi, L., Khalij, L., Gautrelet, C., Shi, C. and Benasciutti, D. (2024). Two-phase optimized experimental design for fatigue limit testing. Probabilistic Engineering Mechanics 75 103551. https://doi.org/10.1016/j.probengmech.2023.103551.
[58] 
Silvapulle, M. J. (1981). On the existence of maximum likelihood estimators for the binomial response models. Journal of the Royal Statistical Society. Series B (Methodological) 310–313. MR0637943
[59] 
Sørensen, C. B., Adams, T. B., Pedersen, E. R., Nielsen, J. and Schmidt, J. H. (2023). AMTAS${^{TM}}$ and user-operated smartphone research application audiometry—An evaluation study. Plos one 18(9) 0291412.
[60] 
Stylianou, M. and Flournoy, N. (2002). Dose Finding Using the Biased Coin Up-and-down Design and Isotonic Regression. Biometrics 58(1) 171–177. https://doi.org/10.1111/j.0006-341X.2002.00171.x. MR1891376
[61] 
Takano, T., Yoshinari, M., Sakurai, K. and Ueda, T. (2024). Cyclic Fatigue Properties of Titanium Alloys for Application in Dental Implants. The Bulletin of Tokyo Dental College 2023-0025.
[62] 
Taleb, N. N. (2001) Fooled by Randomness. Random House, New York.
[63] 
Treutwein, B. (1995). Minireview: adaptive psychophysical procedures. Vision Res. 35 2503–2522.
[64] 
Tsutakawa, R. K. (1967). Asymptotic Properties of the Block Up-and-down Method in Bio-assay. Ann. Math. Stat. 38 1822–1828. https://doi.org/10.1214/aoms/1177698615. MR0217951
[65] 
von Békésy, G. (1947). A new audiometer. Acta Oto.Laryn. 35 411–422.
[66] 
Watson, A. B. and Pelli, D. G. (1983). Quest: A Bayesian adaptive psychometric method. Perception & Psychophysics 33 113–120.
[67] 
Wetherill, G. B. (1963). Sequential estimation of quantal response curves. J Roy. Stat. Soc. B 25 1–48.
[68] 
Wetherill, G. B. and Levitt, H. (1965). Sequential estimation of on a psychometric function. Brit. J Math. Stat. Psych. 18 1–10.
[69] 
Woolf, B. (1955). On Estimating the Relation between blood group and disease. Annals of Human Genetics 19 251–253.
[70] 
Wu, C. F. J. (1985). Efficient Sequential Designs with Binary Data. Journal of the American Statistical Association 80(392) 974–984. MR0819603
[71] 
Zhao, H., Li, X., Tang, N., Jiang, X., Guo, Z. and Lin, H. (2018). Dielectric properties of fluoronitriles/CO2 and SF6/N2 mixtures as a possible SF6-substitute gas. IEEE Transactions on Dielectrics and Electrical Insulation 25(4) 1332–1339. https://doi.org/10.1109/TDEI.2018.007139.

Full article PDF XML
Full article PDF XML

Copyright
© 2024 New England Statistical Society
Open access article under the CC BY license.

Keywords
Adaptive designs Dose-finding Up-and-Down Staircase method

Metrics
since December 2021
187

Article info
views

85

Full article
views

62

PDF
downloads

11

XML
downloads


Share


RSS