Behavioral research often relies on the analysis of timed-event sequential data, where multiple annotators identify and categorize temporally bounded events. It is important to assess the reliability of such data by evaluating the extent to which annotators agree about event identification as well as event categorization. We provide a toolbox to carry out such an analysis, STEADY (Sequential Timed-Event Annotated Data reliabilitY), building on the EasyDIAg toolbox developed by Holle and Rein (2015). EasyDIAg focuses mainly on the evaluation of inter-annotator agreement in terms of event categorization. We extend this framework in two ways. First, we provide a more detailed analysis of event identification, distinguishing types of pairing failures and introducing coder- and label-specific pairing ratios. Second, we incorporate annotator confidence scores as an additional measure of reliability. STEADY is implemented as an open-source R script. We illustrate its use in a step-by-step tutorial in the context of a real-world use case. The STEADY toolbox offers researchers a comprehensive and transparent framework for assessing, visualizing, and improving the inter-annotator agreement of annotations in timed-event
sequential data.