Hong Kong Customs & Excise Department
In recent years, the Hong Kong Syndicate Crimes Investigation Bureau of the Customs and Excise Department (SCIB-C&ED) has seen a sharp rise in illegal sea activities especially smuggling through the sea border. SCIB is a dedicated unit of the Hong Kong C&ED that protects Hong Kong borders against illegal activities, particularly smuggling activities. However, smuggling rings in Hong Kong have changed their primary modus operandi to smuggling by the sea channel especially under extremely bad weather and darkness. In 2022 alone, the SCIB registered over 50 sea smuggling cases, seized items worth $1 billion, and arrested over 100 persons. The existing sea surveillance system in Hong Kong relies on the radar/AIS and it is difficult to distinguish between legitimate maritime traffic and smuggling vessels, especially for wooden/fiber boats with the sea radar. On the other hand, the coastal surveillance is difficult in extremely bad weather conditions and darkness, such as intense fog, clouds, and heavy wind. Therefore, it is imperative to embrace advanced technologies such as hyperspectral imaging for efficient, automated, and real-time sea surveillance in different weather and lighting conditions to safeguard the safety and economy of the Hong Kong coastal borders. A few related multi-spectral surveillance systems are in the market; however, the solutions have disadvantages that render their use challenging to meet the requirements of the C&ED's smart custom effectively. The existing solutions have separate optical paths (individually for each camera), which may cause significant misalignments when fusing images. It needs an extra cost to perform the image pre-processing and sacrifice the image resolution. As such, using the existing solutions may incur longer processing time, lower frame rate, low image quality, high latency, and extensive computational cost. In this project, we will leverage the multi-spectral imaging, high-performance digital signal processing, co-axis optics, edge computing and artificial intelligence technologies to efficiently develop a multi-spectral image fusion solution for efficient surveillance of the coastal area. We will develop an efficient image fusion digital signal processing to combine the images from three multi-spectral cameras (Visible, SWIR, and LWIR) into a single image whose image quality is further enhanced by advanced image enhancement modules in our system. The enhanced fused image provides an overall high-quality image with useful visual information that is could not be obtained from the cameras individually. Each camera module covers a particular portion of the light spectrum. The visible camera provides clear and sharp color images in good lighting conditions. On the other hand, the SWIR and the LWIR provide clear images in foggy or smoky and dark conditions, respectively. We propose an efficient co-axis optical design that allows the three cameras to share the same optical path for better image alignment and eliminates the need for image reorientation, which can cause image distortion and loss of optical information. The co-axis solution is an optical system that utilizes a tailor-made dichromic mirror that splits and directs different light spectra entering different light paths to the individual cameras that sense the corresponding light spectra. Furthermore, we will develop an efficient FPGA (Field Programmable Gate Arrays) architecture for fast and precise image acquisition and synchronization from the three-camera set. The FPGA will also be used for fast and stable zooming control of the camera systems using a PID-based feedback control system. In addition, we will develop an AI-based object tracking architecture in the FPGA to tracking selected objects efficiently and automatically, with low latency (fast) and high accuracy. The FPGA serves as a hardware accelerator (edge computing device) in the system for high performance, low latency, low power consumption, and precise control. With the auto tracking feature, the system provides a means of alerting users and the capability of tracking a desired detected object. For the control interface, the system provides a command and control software interface to allow the operators manage and control the operation of the surveillance system. The interface will provide options and control commands for the operator so that the operator can direct the operation of the system if desired. Through the interface, the operator may command the system to point to a particular direction, track a particular object, stay fixed at a given location, continue auto-scan, and several other options. The operator may use the interface to analyze or disseminate the information provided by the system. Furthermore, the provided interface could notify and alert the operator about occurrence of certain events or alarms set by the operator. It provides a user friendly and simplified interface for easy and efficient operation and control the system for operators. With the support of the Hong Kong C&ED, we will upgrade the existing sea surveillance in Hong Kong to a high-performance, intelligent, and real-time sea surveillance system that functions efficiently and reliably under different environmental conditions to fully safeguard the Hong Kong border against smuggling and other illegal coastal activities.