DESIGNING A PIPELINE WITH BIG DATA TECHNOLOGIES FOR BORDER SECURITY

Fatih Aydemir, Aydın Çetin
1.579 343

Abstract


Developing technology, communication and transportation facilities have led borders to the gradual disappearance. The incidents taking place in different countries go beyond the limits and being felt by large audiences. Solutions to rapidly increasing terrorist attacks and internal turmoil caused security problems in recent years which have become very complicated. Protecting and controlling the borders, finding rapid and comprehensive solutions to illegal border passing problems have become a major problem in many countries. When considering the scope of the border security, it would be understood to examine the data which comes at different times from different sources and types in real time. This can be achieved by a collaboration of different principles. In addition, when large-scale unstructured data analysis also considered for a comprehensive solution, it would be a rational method to develop a pipeline using the Big Data technologies. In this study, we aim to achieve cost-effective, robust, scalable and flexible system to solve the border security problems. The use of Lambda Architecture which provides real time data processing and batch processing capabilities was investigated in border security applications. System development was explained and all information on its principles was provided.

Full Text:

PDF

References


Aksu, M. and Turhan, F. “New Threats, Expansion of Security Dimensions and Human Security”, International Journal of Alanya Faculty of Business, Vol. 4, 69-80, 2012

Eker, G. and Yılmaz, G. “Providing Environmental Security Using Wireless Sensor Networks”, TBV BBMD, 64-71, 2013

Demchenko, Y., de Laat, C. and Membrey, P, “Defining architecture components of the Big Data Ecosystem”, Collaboration Technologies and Systems (CTS), 2014, 104-112

Engel, Y., and Opher, E., “Towards proactive event-driven computing”, 5th ACM international conference on Distributed event-based system (DEBS '11), 2014, 125-136

Kreps, J., Narkhede, N. and Rao, J., “Kafka: A distributed messaging system for log processing”, Proceedings of 6th International Workshop on Networking Meets Databases, 2011

Zaharia, M., Das, T., Li, H., Shenker, S. and Stoica, I., “Discretized streams: an efficient and fault-tolerant model for stream processing on large clusters”, 4th USENIX conference on Hot Topics in Cloud Computing, 2012

Chebotko Kashlev, A. and Shiyong, L., “A Big Data Modeling Methodology for Apache Cassandra”, Big Data (BigData Congress), IEEE International Congress, 2015, 238-245

Nodejs, https://nodejs.org/en/about/

Cesiumjs, https://cesiumjs.org/

The Reactive Manifesto, http://www.reactivemanifesto.org/

Nathan Marz and James Warren, Principles and best practices of scalable realtime data systems, Manning, 2015, 328 pages

Spark Streaming + Kafka Integration, http://spark.apache.org/docs/latest/streaming-kafka-integration.html

Kiran, M., Murphy, P., Monga, I., Dugan, J. and Baveja, S., “Lambda architecture for cost-effective batch and speed big data processing.” Big Data (Big Data Congress), IEEE International Conference, 2015, 2785-2792