Setting up research oriented wiki server
I spent this summer setting up a research oriented Wiki server to aid China and North Korea area studies researchers in their work. It brings together declassified primary source documents as well satellite imagery, on-the-ground imagery, technically manuals, and secondary research documents in a single resource. It uses the same software as Wikipedia, called MediaWiki, which allows for easy editing and sharing of data.
The ultimate purpose of the Research Wiki is threefold. First it aims to create a site in which satellite imagery, primary source documents, video media, and all related data from any form of media can be easily linked together to create better contextualization and data integration. Second it aims to establish a ‘common knowledge’ baseline in the vein of Wikipedia, but for very specialized regional studies. Third, it creates a research environment in which information from a number of disciplines can be easily linked together in the support of the common knowledge baseline objective.
The wiki-style format of data entry and collection allows users with varying expertise to collaborate and better everybody’s basic knowledge. All scholars and academics retain proprietary knowledge that they don’t want to share, but most have pieces of data and information they write off as common, basic knowledge, yet this knowledge is not known by other scholars in the same field and can help built a better common context and view of a region or situation.
Especially as the security study field continues to expand into numerous other disciplines, common knowledge of potentially obscure topics such as electrical generation, agricultural methods and reform, and propaganda themes can help to better contextualize security situations and ultimately lead to deeper research and analyses.
While there are still a few technical challenges to overcome, the tentatively named Research Wiki is currently up and running. It so far hosts declassified documents dealing with China, North Korea, Libya, and the Soviet Union, satellite imagery, on-the-ground imagery, research data and a host of other supporting documents and data sources. It is not limited to these and, like any other Wiki format site, can expand as far as the knowledge and contributions of its user base. It has been opened up to a specific group of researchers who are slowly adding their own knowledge and data sources to it and, if everything stays on schedule, it would optimally be opened to the general body of Security Studies students within a semester.
How did you come up with your project?
This project is basically an evolution of a personal project I started two years ago in an attempt to better organize my myriad notes. I was working on a contract with a think tank doing strategic missile analysis and building a research blog with a close friend, and between the two had an enormous amount of personal research notes strewn across every medium possible. As well, I had almost every type of media source to manage. I was handling videos of missile launches, on-the-ground imagery of weapons systems, satellite imagery of basing and support structures, declassified documents detailing what everything looked like in the 70s and 80s, and then the equivalent of all of this for hydroelectric production and petroleum resource management in the DPRK.
It was a nightmare to manage, so I searched online for a good management solution and decided to try out Mediawiki. I built a tiny wiki server on a Raspberry Pi microcomputer and suddenly could basically have all media related to one topic in one place in a format that matched how I mentally organized my data. I could have documents detailing the construction of DPRK hydroplants loaded on a page with pictures of the modern equivalents to match, then link to notes on a recently constructed dam with supporting video and satellite imagery to help me get a feel for how everything fit together. Essentially this project is a way of pulling together data from a number of sources and having all those sources right in front of a user, in addition to the knowledge pulled from those sources.
Did anything surprise you?
It turns out academics, who are notoriously defensive of their territories and information, can be very forthcoming and helpful when putting together projects. It helped that the initial testing group I am working with all know me personally, but most of them do not know each other, which means they are still taking a small risk or hit by effectively donating some of their own basic knowledge and time for the betterment of North Korea and China research as a whole. Hopefully this trend continues as the wiki is opened to more people, as wiki-type projects without contributors and new data sources tend to die off.
What are two interesting things about this project that the average person doesn’t know?
Two interesting things that people would know offhand is how weirdly difficult it is to get files uploaded and working and how interesting a historical intelligence narrative is.
It won’t be as surprisingly to people with computer science or technical backgrounds, but getting files and systems to actually function as one ideally expects them to do is frustrating. When I was running my small prototype machine, I could employ weird workarounds and ad-hoc solutions since I was the only person using the information. However, all solutions and setups for this version have to be both functional and user friendly, and having some obscure means of setting up notes and documentation is not helpful for anybody. However figuring out how to get things running smoothly has definitely had the bonus effect of giving me more practice with Linux based servers.
Going through declassified documents from previous decades provides really interesting data, not only because of the actual information the documents contain, but because of the narrative they build. Historical documents can help contextualize what certain parts of the intelligence communities were thinking in the background of major world events, including times when they were incorrect in their assessment. It’s interesting to see portions of the intelligence community struggling with problems and sometimes failing to solve them, it’s a reminder that even the agencies make mistakes and have trouble sometimes.
The ultimate purpose of the Research Wiki is threefold. First it aims to create a site in which satellite imagery, primary source documents, video media, and all related data from any form of media can be easily linked together to create better contextualization and data integration. Second it aims to establish a ‘common knowledge’ baseline in the vein of Wikipedia, but for very specialized regional studies. Third, it creates a research environment in which information from a number of disciplines can be easily linked together in the support of the common knowledge baseline objective.
The wiki-style format of data entry and collection allows users with varying expertise to collaborate and better everybody’s basic knowledge. All scholars and academics retain proprietary knowledge that they don’t want to share, but most have pieces of data and information they write off as common, basic knowledge, yet this knowledge is not known by other scholars in the same field and can help built a better common context and view of a region or situation.
Especially as the security study field continues to expand into numerous other disciplines, common knowledge of potentially obscure topics such as electrical generation, agricultural methods and reform, and propaganda themes can help to better contextualize security situations and ultimately lead to deeper research and analyses.
While there are still a few technical challenges to overcome, the tentatively named Research Wiki is currently up and running. It so far hosts declassified documents dealing with China, North Korea, Libya, and the Soviet Union, satellite imagery, on-the-ground imagery, research data and a host of other supporting documents and data sources. It is not limited to these and, like any other Wiki format site, can expand as far as the knowledge and contributions of its user base. It has been opened up to a specific group of researchers who are slowly adding their own knowledge and data sources to it and, if everything stays on schedule, it would optimally be opened to the general body of Security Studies students within a semester.
How did you come up with your project?
This project is basically an evolution of a personal project I started two years ago in an attempt to better organize my myriad notes. I was working on a contract with a think tank doing strategic missile analysis and building a research blog with a close friend, and between the two had an enormous amount of personal research notes strewn across every medium possible. As well, I had almost every type of media source to manage. I was handling videos of missile launches, on-the-ground imagery of weapons systems, satellite imagery of basing and support structures, declassified documents detailing what everything looked like in the 70s and 80s, and then the equivalent of all of this for hydroelectric production and petroleum resource management in the DPRK.
It was a nightmare to manage, so I searched online for a good management solution and decided to try out Mediawiki. I built a tiny wiki server on a Raspberry Pi microcomputer and suddenly could basically have all media related to one topic in one place in a format that matched how I mentally organized my data. I could have documents detailing the construction of DPRK hydroplants loaded on a page with pictures of the modern equivalents to match, then link to notes on a recently constructed dam with supporting video and satellite imagery to help me get a feel for how everything fit together. Essentially this project is a way of pulling together data from a number of sources and having all those sources right in front of a user, in addition to the knowledge pulled from those sources.
Did anything surprise you?
It turns out academics, who are notoriously defensive of their territories and information, can be very forthcoming and helpful when putting together projects. It helped that the initial testing group I am working with all know me personally, but most of them do not know each other, which means they are still taking a small risk or hit by effectively donating some of their own basic knowledge and time for the betterment of North Korea and China research as a whole. Hopefully this trend continues as the wiki is opened to more people, as wiki-type projects without contributors and new data sources tend to die off.
What are two interesting things about this project that the average person doesn’t know?
Two interesting things that people would know offhand is how weirdly difficult it is to get files uploaded and working and how interesting a historical intelligence narrative is.
It won’t be as surprisingly to people with computer science or technical backgrounds, but getting files and systems to actually function as one ideally expects them to do is frustrating. When I was running my small prototype machine, I could employ weird workarounds and ad-hoc solutions since I was the only person using the information. However, all solutions and setups for this version have to be both functional and user friendly, and having some obscure means of setting up notes and documentation is not helpful for anybody. However figuring out how to get things running smoothly has definitely had the bonus effect of giving me more practice with Linux based servers.
Going through declassified documents from previous decades provides really interesting data, not only because of the actual information the documents contain, but because of the narrative they build. Historical documents can help contextualize what certain parts of the intelligence communities were thinking in the background of major world events, including times when they were incorrect in their assessment. It’s interesting to see portions of the intelligence community struggling with problems and sometimes failing to solve them, it’s a reminder that even the agencies make mistakes and have trouble sometimes.