Research Spotlight: Calibration and Development of Safety Performance Functions for New Jersey

In 2019, a team of researchers from New York University and Rutgers University examined ways to calibrate and develop Safety Performance Functions (SPFs) to be utilized specifically to address conditions on New Jersey roadways. SPFs are crash prediction models or mathematical functions informed by data on road design. These data include, but are not limited to, lane and shoulder widths, the radius of the curves, and the presence of traffic control devices and turn lanes. With these data, SPFs help those tasked with road design and improvement to build roads and implement upgrades that maximize safety.

The Highway Safety Manual (HSM) presents SPFs developed using historic crash data collected from several states over several years at sites of the same facility type. These SPFs data cannot be transferred to other locations because of expected differences in environment and geographic characteristics, crash reporting policies and even local road regulations. To help SPFs better reflect local conditions and observed data, one of two strategies is usually undertaken to fine-tune SPFs:  calibrating the SPFs provided in the HSM so as to fully leverage these data or developing location-specific SPFs regardless of the predictive modeling framework included in the HSM.

The research team, led by Dr. Kaan Ozbay (of NYU’s Tandon School of Engineering), chose to pursue both of these strategies. The research report, Calibration/Development of Safety Performance Functions for New Jersey, can be found here. A webinar highlighting the research and findings can be found here.  A monograph, supported by the NJDOT funded study and partially by C2SMART, a Tier 1 UTC led by NYU and funded by the USDOT, was also recently published and can be found here.

C2SMART Webinar highlighted the research methods, findings, challenges and technology transfer efforts of the NYU-Rutgers team for this NJDOT funded research project.

SPFs can be utilized at several levels. At the network level, researchers and engineers use SPFs to identify locations with promise for improvement. SPFs can be used to predict how safety treatments will affect the likelihood of crashes based on traffic volume and facility type. SPFs can be used to influence project level design by showing the average predicted crash frequency for an existing road design, for alternate designs, and for brand-new roads.

SPFs also can be used to evaluate different engineering treatments. In this case, engineers and researchers return to a site where a safety countermeasure has been installed to collect and analyze data to see how the change has affected crash frequency. They examine before and after conditions and measure if the prediction made using the SPF was accurate or needs improvement (Srinivasan & Bauer, 2013). In the end, SPFs are only as good as the data used in their development.

NJDOT and the NYU-Rutgers team set out to calibrate SPFs using New Jersey’s roadway features, traffic volumes and crash data, and if necessary, to create new SPFs that reflect conditions in the state. The facility types considered for this research project included segments and intersections of rural two-lane two-way, rural multilane, and urban and suburban roads. In examining these datasets, the researchers identified areas where data processing improvements could be made to enhance the quality or efficiency in use of the data in addition to pursuing the stated goal of developing New Jersey-specific SPFs.

For example, utilizing the data provided by NJDOT, the research team developed methods for processing a Roadway Features Database of different kinds of road facilities. The researchers utilized the Straight Line Diagrams (SLD) database, which offers extensive information about the tens of thousands of miles of roadways in New Jersey, but observed issues and errors in the SLD database that required corrections. For example, the research team utilized Google Maps and Google Street View to conduct a manual data extraction process to verify information in the SLD database (e.g., confirm whether an intersection was an overpass, number of lanes, directionality) and extract missing variables, such as the number of left and right turn lanes at intersections, lighting conditions, and signalization needed for the analysis.

The research team using Google Street View to identify missing data points.

The research team also needed to develop programming code to correctly identify the type and location of intersections and effectively work with available data. The team developed a novel “clustering-based approach” to address the absence of horizontal curvature data using GIS centerline maps.

Utilizing Google Maps (Left) and the state’s Straight Line Database (Right), researchers were able to identify missing paths in the database that contributed to inconsistent data.

Police reports of crashes often have missing geographic identifiers which complicates analytical work such as whether crashes were intersection-related. In NJ, police are equipped with GPS devices to record crash coordinates but this crash information is somewhat low in the raw crash databases before post-processing by NJDOT. The researchers employed corrective methods and drew upon other NJ GIS maps to provide missing locations (e.g., Standard Route Identification or milepost).

The processing challenges for roadway features, traffic volumes and crashes encountered by the research team suggest the types of steps that can be taken to standardize and streamline data collection and processing to secure better inputs for future SPF updates. Novel data extraction methods will be needed to minimize labor time and improve accuracy of data; accurate crash data is integral to employing these methods.

The research team modified the spreadsheets developed by the HSM and used by the NJDOT staff. The calculated calibration factors and the developed SPFs are embedded in these spreadsheets. The users can now select whether to use the HSM SPFs with the calculated calibration factors or the New Jersey-specific SPF in their analyses

The researchers’ data processing and calibration efforts sought to ensure that the predictive models reflect New Jersey road conditions that are not directly reflected in the Highway Safety Manual. The adoption of this data-driven approach can make it possible to capture information about localized conditions but significant expertise is required to carry out calibration and development analyses. With more research—and improved data collection processes over time —the calibration and development of SPFs holds promise for helping New Jersey improve road safety.


Resources

Bartin, B., Ozbay, K., & Xu, C. (2022). Safety Performance Functions for Two-Lane Urban Arterial Segments. Retrieved from https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4175945

C2SMART. (2020, September 23). Webinar: Bekir Bartin, Calibration and Development of Safety Performance Functions for New Jersey . Retrieved from YouTube: https://youtu.be/IRalyvjDaFM

Ozbay, K., Nassif, H., Bartin, B., Xu, C., & Bhattacharyya, A. (2019). Calibration/Development of Safety Performance Functions for New Jersey [Final Report]. New Jersey Department of Transportation Bureau of Research. Retrieved from https://www.njdottechtransfer.net/wp-content/uploads/2020/07/FHWA-NJ-2019-007.pdf

Ozbay, K., Nassif, H., Bartin, B., Xu, C., & Bhattacharyya, A. (2019). Calibration/Development of Safety Performance Functions for New Jersey [Tech Brief]. Rutgers University. Department of Civil & Environmental Engineering; New York University. Tandon School of Engineering. Retrieved from https://www.njdottechtransfer.net/wp-content/uploads/2020/07/FHWA-NJ-2019-007-TB.pdf

Srinivasan, R., & Bauer, K. M. (2013). Safety Performance Function Development Guide: Developing Jurisdiction-Specific SPFs. The University of North Carolina, Highway Safety Research Center. Retrieved from https://rosap.ntl.bts.gov/view/dot/49505

Zone for AI to look for trespassing at railroad crossing

Research Spotlight: Exploring the Use of Artificial Intelligence to Improve Railroad Safety

Partnering with the Federal Railroad Administration, New Jersey Transit and New Jersey Department of Transportation (NJDOT), a research team at Rutgers University is using artificial intelligence (AI) techniques to analyze rail crossing safety issues. Utilizing closed-circuit television (CCTV) cameras installed at rail crossings, a team of Rutgers researchers, Asim Zaman, Xiang Liu, Zhipeng Zhang, and Jinxuan Xu, have developed and refined an AI-aided framework for detection of railroad trespassing events to identify the behavior of trespassers and capture video of infractions.  The system uses an object detection algorithm to efficiently observe and process video data into a single dataset.

Rail trespassing is a significant safety concern resulting in injuries and deaths throughout the country, with the number of such incidents increasing over the past decade. Following passage of the 2015 Fixing America’s Surface Transportation (FAST) Act that mandated the installation of cameras along passenger rail lines, transportation agencies have installed CCTV cameras at rail crossings across the country.  Historically, only through recorded injuries and fatalities were railroads and transportation agencies able to identify crossings with trespassing issues. This analysis did not integrate information on near misses or live conditions at the crossing. Cameras could record this data, but reviewing the video would be a laborious task that required a significant resource commitment and could lead to missed trespassing events due to observer fatigue.

Zaman, Liu, Zhang, and Xu saw this problem as an opportunity to put AI techniques to work and make effective use of the available video and automate the observational process in a more systematic way. After utilizing AI for basic video analysis in a prior study, the researchers theorized that they could train an AI and deep learning to analyze the videos from these crossings and identify all trespassing events.

Working with NJDOT and NJ TRANSIT, they gained access to video footage from a crossing in Ramsey, NJ.  Using a deep learning-based detection method named You Only Look Once or YOLO, their AI-framework detected trespassings, differentiated the types of violators, and generated clips to review. The tool identified a trespass only when the signal lights and crossing gates were active and tracked objects that changed from image to image in the defined space of the right-of-way. Figure 1 depicts the key steps in the process for application of AI in the analysis of live video stream or archived surveillance video.

Figure 1. General YOLO-Based Framework for Railroad Trespass Detection illustrates a step-by-step process involving AI algorithm configurations, YOLO-aided detection, and how trespassing detection incidents are saved and recorded to a database for more intensive analysis and characterization (e.g., trespasser type, day, time, weather, etc.)

The researchers applied AI review to 1,632 hours of video and 68 days of monitoring. They discovered 3,004 instances of trespassing, an average of 44 per day and nearly twice an hour. The researchers were able to demonstrate how the captured incidents could be used to formulate a demographic profile of trespassers (Figure 2) and better examine the environmental context leading to trespassing events to inform the selection and design of safety countermeasures (Figure 3).

Figure 2: Similar to patterns found in studies of rail trespassing fatalities, trespassing pedestrians were more likely to be male than female. Source: Zhang et al
Figure 3: Trespassing events were characterized by the gate angle and timing before/after a train pass to isolate context of risky behavior. Source: Zhang et. al

A significant innovation from this research has been the production of the video clip that shows when and how the trespass event occurred; the ability to visually review the precise moment reduces overall data storage and the time needed performing labor-intensive reviews. (Zhang, Zaman, Xu, & Liu, 2022)

With the efficient assembly and analysis of video big data through AI techniques, agencies have an opportunity, as never before, to observe the patterns of trespassing. Extending this AI research method to multiple locations holds promise for perfecting the efficiency and accuracy in application of AI techniques in various lighting, weather and other environmental conditions and, more generally, to building a deeper understanding of the environmental context contributing to trespassing behaviors.

In fact, the success of this AI-aided Railroad Trespassing Tool has led to new opportunities to demonstrate its use. The researchers have already expanded their research to more crossings in New Jersey and into North Carolina and Virginia. (Bruno, 2022) The Federal Railroad Administration has also awarded the research team a $582,859 Consolidated Rail Infrastructure and Safety Improvements Grant to support the technology’s deployment at five at-grade crossings in New Jersey, Connecticut, Massachusetts, and Louisiana. (U.S. DOT, Federal Railroad Administration, 2021) Rutgers University and Amtrak have provided a 42 percent match of the funding.

The program’s expansion in more places may lead to further improvements in the precision and quality of the AI detection data and methods.  The researchers speculate that this technology could integrate with Positive Train Control (PTC) systems and highway Intelligent Transportation Systems (ITS). (Zhang, Zaman, Xu, & Liu, 2022) This merging of technologies could revolutionize railroad safety. To read more about this study and methodology, see this April 2022 Accident Analysis & Prevention article.

References

Bruno, G. (2022, June 22). Rutgers Researchers Create Artificial Intelligence-Aided Railroad Trespassing Detection Tool. Retrieved from https://www.rutgers.edu/news/rutgers-researchers-create-artificial-intelligence-aided-railroad-trespassing-detection-tool

NJDOT Technology Transfer. (2021, November 8). How Automated Video Analytics Can Make NJ’s Transportation Network Safer and More Efficient. Retrieved from https://www.njdottechtransfer.net/2021/11/08/automated-video-analytics/

Tran, A. (n.d.). Artificial Intelligence-Aided Railroad Trespassing Data Analytics: Artificial Intelligence-Aided Railroad Trespassing Data Analytics:.

United States Department of Transportation: Federal Railroad Administration. (2021). Consolidated Rail Infrastructure and Safety Improvements (CRISI) Program: FY2021 Selections. Retrieved from https://railroads.dot.gov/elibrary/consolidated-rail-infrastructure-and-safety-improvements-crisi-program-fy2021-selections

Zaman, A., Ren, B., & Liu, X. (2019). Artificial Intelligence-Aided Automated Detection of Railroad Trespassing. Journal of the Transportation Research Board, 25-37.

Zhang, Z., Zaman, A., Xu, J., & Liu, X. (2022). Artificial intelligence-aided railroad trespassing detection and data analytics: Methodology and a case study. Accident Analysis & Prevention.