Welcome to visit Vikylin.com.
We sincerely expect to build long-term relationship with you.
Costomer Service +86 75523035863
Keywords: Big Data Intelligence Analysis
One, what is big data?
For the definition of big data, there were different voices for different careers at that time. Some people say that big data is a large amount of data. It used to be a TB rating. Today it is a PB rating. Others say that big data is a new skill represented by Hadoop. It can handle massive amounts of data. These kinds of arguments seem a bit one-sided, one focuses on data, one focuses on skills. McKinsey said “4V” to the big data industry and made a consensus on big data: “Big data means that the size of the data set exceeds the capabilities of the typical database software and things that are available today. New skills and new talents for storing, aggregating, handling these big data, and in-depth analysis of the data are growing rapidly, just as Moore's Law predicts the growth rate of accounting chips." McKinsey's definition covers data and skills, however, With the development of big data, such a definition cannot fully interpret the internality of big data. We say that big data does not only require data and skills, but more importantly, it can provide very good service. Big data can analyze the massive data in depth, and then make guesses about the trend of things. This is also the central location of big data. Big data can apply mathematical algorithms to massive data to guess the possibility of job seizures.
The book “The Day of Big Data” pointed out that the essence of big data lies in the three changes that we make when we analyze the data. These changes will help us deepen our understanding of big data.
First, in the era of big data, we can analyze the entire data set, rather than sampling data. Mutations in the data can be mutated and together they can compensate for the lack of algorithms. Look at the following examples. In the grammar view of the Word program, there is a simple algorithm. When the amount of data is only 5 million, the algorithm performs poorly, but when the amount of data reaches 1 billion level, the algorithm performs best. On the contrary, there is a cluttered algorithm that performs best with 5 million data volumes, but it is less effective than a simple algorithm with a data volume of 1 billion levels.
Second, there is so much data under big data that we are no longer keen to seek accuracy. In the era of the lack of information, we generally seek the accuracy of the data; in the era of big data, the amount of data is so large and the data types are so complicated that we cannot ensure that each data is accurate, but we only need to ensure that the larger data is Accurately, it will not affect the reliability of the analysis results.
Third, in the era of big data, we are more concerned about relationships, not causal links. Wal-Mart's analysis system found that there is a certain relationship between beer and diaper sales. Based on the results of the analysis, Wal-Mart put together diapers and beer, two kinds of unrelated products, the results of the unique discovery of diapers and beer sales have increased. It turns out that American women usually take care of their children at home, so they often tell their husbands to buy diapers for their children on their way home from work, and the husband buys diapers and buys his own favorite beer. In this case, we have found its factors, but more time, we can not find the factors, and in fact we do not need to care for its factors, because from the analytical results of the relationship, we can get from beneficial.
Second, the status of big data development
Big data is now the hottest skill in the country. In 2012, the Ministry of Science and Technology’s “China’s cloud science and technology launched the 12th Five-Year Plan” and the Ministry of Industry and Information Technology’s “12th Five-Year Plan” on the Internet of Things, both taking big data skills as One point is supported.
In the area of IT, the development of big data has been properly seasoned. If Google uses the instructions of more than 3 billion users to successfully predict the spread of the flu, the use of trillions of corpora will provide users with relatively accurate translations; Amazon will use historical information to guess books that are of interest to users; Taobao is accurate based on users' shopping practices. Push ads; and so on.
However, in the area of security, big data is still in its infancy and exploration.
First and foremost, the security profession is gradually entering the era of big data. With the continuous development of the urban process and the continuous deepening of informatization construction, data is growing rapidly at the level of several levels. Traditional systems or things can no longer effectively handle such massive amounts of data. For example, the traffic bayonet data used to be of the order of 10 million. Today's situation is: One district/county bayonet data can reach one billion grade, and a prefecture-level city's one-year bayonet data can even reach ten billion grades. The data of one province is even greater. Faced with such endless data, the traditional system seems to be incapable of doing anything. Even if a simple inquiry instruction is used, the response time will become very slow, not to mention the functions of profiling and accounting. Together, increasing users have made higher requests for big data, such as public security users. They have grasped a lot of data, are of various types, and have large amounts of data. Their requests can be analyzed through mass data to arrive at the effect of guessing and warning. The public security service can be post-analyzed and guessed and changed beforehand.
Second, some security companies are touching big data and have begun to explore and use it. As early as 2012, Hikvision entered big data, developed and optimized big data processing solutions according to Hadoop, and was satisfied with the request for efficient processing of massive data. At that time, Hikvision's products based on big data skills were: video cloud storage, which could satisfy the storage of 100 PB data; video picture information database, which could quickly retrieve data for a large number of case cases; traffic data could be used for big data channels. Massive bayonet data are quickly retrieved, intelligently analyzed, and analysed by accounting. Some of the functions of judgements can be used for investigation and early warning of criminal cases. In addition, security companies like Bocom, Yushi, etc. are also catching up with the pace of big data.
Third, security big data center skills analysis
The development of big data in the IT area has been appropriately sophisticated, and many skills in it can be applied to the security field. However, security occupations exist in areas that are not the same as IT occupations, mainly data types. In the IT profession, the analysis targets of big data are generally logs, user information, page indexes, and other data. They are structured data that can be identified by the accounting machine. In the security profession, the goals that big data needs to be analyzed are video, drawings, Unstructured data such as audio, accounting machine can not directly analyze the data, but need to obtain the structural information between them, and then analyze.
The basic skills of big data can be learned from the IT category to the security category. These skills include the following: 1. Distributed file system, serving as a massive data storage, storing data on multiple independent devices, and adopting an expandable system The architecture uses multiple storage servers to manage the storage load and uses metadata servers to locate the storage information. It not only improves the system's robustness, availability, and access power, but also facilitates expansion. Second, distributed database, and column-oriented real-time Distributed database, suitable for building high-concurrency and low-delay online data service system for storing coarse-grained structured data; Third, distributed accounting, act as a problem that requires a very infinite accounting to deal with the problem of dividends, many small Some of them are then assigned to a lot of accounting machines for processing, and finally the results are summed up to get the final results; Fourth, the full-text search engine, as a secure, reliable and rapid real-time retrieval of massive data; Fifth, memory accounting, distributed Memory accounting can increase the amount of data quickly Analysis process; six flow accounting, as the streaming media data analysis process. Based on these skills, the structured data can be quickly processed to handle the power of mass data processing.
However, as mentioned above, the most data in the security profession are not structured data. However, structured data is extracted from these unstructured data and is the key point that needs to be dealt with. The structured information that can be obtained in video drawings includes the following: 1. Characteristics of people, vehicles, and objects. Human characteristics include gender, age group, height, body type, skin color, whether they wear glasses, hairstyles, and clothing features. , With objects, the car features information including license plate number, license plate color, license plate type, vehicle type, body color, car logo, on-board personnel information, etc. The object's characteristic information includes the article's color, shape, size, texture characteristics, etc.; Second, practice information, such as through the alert surface, entering/leaving area, regional aggression, personnel paralysis, personnel gathering. When these data are obtained, further analysis can be conducted, such as the analysis of the vehicle's orbit, and an analysis of people's anomalous practices. Therefore, intelligent profiling skills are particularly important in security big data and are the basis for accomplishing big data security.
After combining a lot of data, you need to explore the depth value of the data. The real value of the data is like the icebergs in the ocean. At first glance, only the tip of the iceberg can be seen, while some of the vast majority are hidden under the surface. Guess is the central value location of big data, and depth-related parsing algorithms are necessary to complete the value of big data. The data parsing algorithm is like a drill bit, and the real diamonds need to be discovered from this unique diamond mine of big data.
Fourth, security big data face doubts
With the development of big data, many questions have gradually emerged. The main points are as follows:
(1) Smart parsing skills are not sophisticated. The structuring of video picture data is the basis for the completion of security big data. At that time, the vehicle information acquisition skills of the traffic bayonet were relatively sophisticated, but skills such as human body information acquisition and face matching were not sophisticated.
(2) The use of data will not be deepened. When combined with enough data, how to use the data for speculation analysis, trend analysis, almost at the time of the use of the form of blank. Of course, there are also some examples that can be learned, such as the bayonet big data system, which can conduct in-depth smart judgments on passing data: regional bumps, track analysis, follow-up study, etc., which contributes to the dramatic increase in the power of criminal investigation .
(3) Data sharing is not extensive. In exceptional cases, such as government, public security, and transportation, information islands are ubiquitous. This is mainly due to the formation of doubts in the system. It is not a technical issue. It is difficult for companies to change the status quo. Some of them can only follow the promotion. Make changes.
(4) Normative construction is not all. This is mainly because big data is still in its infancy, and further exploration and testing are needed. In terms of standardized construction, such as data specification specification, interconnection specification specification, data usage form specification specification, etc., need to be continuously summarized and gradually standardized.
V. Security Big Data Development Trend
In the future development of the security profession, big data is bound to occupy an increasingly important position. Facing the doubts in the development process, the primary task at that time was to be able to gradually handle these questions and continuously improve the security big data program.
(1) The skills are different. First and foremost, the structuring of video data, through intelligent skills, can capture features such as people, vehicles, and objects from video drawings. After acquiring and combining these information, video data can be easily retrieved and searched. Depth analysis. When these skills are completed, the power of video data will be greatly increased, and it can be considered that the deepening use of video data lays the foundation. Second, big data processing skills. After the video data is structured, it becomes data that can be recognized by the accounting machine. When the ever-increasing data gathers, the traditional skills or system can no longer be usefully processed. At this moment, it is necessary to use big data skills to carry out these massive data. deal with. Big data skills include distributed file system, distributed database, full-text search engine, distributed accounting, memory accounting, flow accounting, etc. It has excellent reliability, scalability, and processing performance. It can quickly analyze and excavate massive data. Provide a very good service for users.
(2) The business is different. With structured massive video data, and through big data skills, these massive data can be in-depth explored, and speculation and trend analysis can be done. However, related transaction models need to be continuously explored and developed. For example, some public security agencies, video surveillance can only be an auxiliary method at the time, if you can use the big data skills to make predictions and warnings, then video surveillance will become a very important method, after the inspection skills, you can cut Case attack rate, improve the detection rate.
(3) System improvement. More data can be of greater value. In order to be able to combine more data, it is necessary to eliminate information islands, and this is a problem that exists in the government. Of course, under the advancement of talented cities, this situation has changed, and the increasing government is aware of the importance of data sharing. However, there is still a long way to go before realizing the concentration and sharing of big data.
(4) The specification is perfect. The combination of massive data is inseparable from the normalization process. In the process of standardization, it is necessary to consider the following points: First, the data structure specification and specification, which data needs to be structured, structured data how to indicate, how to plan the dictionary specification, How to plan the database tables, etc., through standardized and structured data, all systems can be identified and handled; Second, the specification of data interconnection and interoperability includes how to interconnect and intercommunicate between channels and front-ends, and how to interconnect and intercommunicate channels and channels. The front-end can structure the video data. The back-end can also structure the video data. The front-end and back-end need to cooperate with each other. Then the front-end tells the backend what data is already structured and what data needs further structuring. It needs to be standardized. To regulate; Third, the specification of data use, including the data of the service forms, types, rules and so on. If the big data channel cleans, classifies, and in-depth excavates massive amounts of data, it will need to use supply services for upper-level affairs. Such services need to be provided through a standardized interface.
6. Big data helps the development of security profession
Big data in the security profession will have a process from start to start and experience. At the time of its inception, there were some intelligent profiling skills, and through the big data skills, the processing power of mass data was being questioned; during the period of development, the intelligent profiling skills would continue to be sophisticated and would constantly present different data usage; during the period of experience, Smart parsing skills are appropriately sophisticated and systematically use data in depth. Big Data helps the development of the security profession, mainly in the following aspects:
First, the use of data power continues to increase. Through intelligent profiling skills and big data skills, it is possible to increase the power of video data usage and handle the problems of low power consumption in the past. Increased use of power can make video data more valuable.
Second, the depth of data use. The in-depth use of data can represent the true value of big data, and this can also improve the overall strength of the security system, and make the marginal position of video data closer to the center, so that the competitiveness of the security profession can be improved.
Third, improve the system and specifications. The improvement of norms and systems can further promote the development of big data, and a well-regulated security company will have a stronger voice.
Big data can build a more intelligent system, which can change the status of “re-construction and light use” in the construction of China’s security system, help to further improve some of the central combat effectiveness of public safety, and further reinforce the foundation for social stability.
Big data can be thought of as a way for users to build more intelligent systems and provide more valuable services. In the security profession, the rapid growth of data and the constantly present needs of users indicate that the demand for big data has become increasingly fierce. Together with the increasing number of security companies entering big data, they have begun to explore and use it. The big data in the security category is not the same as the big data in the IT category. It has higher requests for intelligent profiling skills, and intelligent profiling skills are the basis for completing big data security. In addition, it has big data skills, data depth analysis algorithms, etc. There is also the same high request. Of course, big data is still in the security profession is still in its infancy, not experienced big data also face many questions, including intelligent analysis skills are not sophisticated, data use is not deepened, data sharing is not widely, normative construction not all. In the future development, we must first deal with these questions, and continue to improve the security big data program, including skills, services, systems, norms and improve. Only more perfect security big data can show more significant advantages and show greater value. With the continuous development of big data, it will certainly bring a qualitative improvement to the security profession. Big data is the future development trend, it will lead the next security era, let us wait and see