Hostname: page-component-848d4c4894-m9kch Total loading time: 0 Render date: 2024-06-01T14:19:00.456Z Has data issue: false hasContentIssue false
  • English
  • Français

Dynamic elicited priors for updating covert networks

Published online by Cambridge University Press:  15 April 2013

JEFF GILL  and
JOHN R. FREEMAN
JEFF GILL
Affiliation:
Department of Political Science, Department of Biostatistics, and Department of Surgery (Public Health Sciences), Washington University, One Brookings Drive, Seigle Hall, St. Louis, MO 63130, USA (e-mail: jgill@wustl.edu)
JOHN R. FREEMAN
Affiliation:
Department of Political Science, University of Minnesota, 1414 Social Sciences Bldg, 267 19th Avenue South, Minneapolis, MN 55455, USA
Get access Rights & Permissions [Opens in a new window]

Abstract

The study of covert networks is plagued by the fact that individuals conceal their attributes and associations. To address this problem, we develop a technology for eliciting this information from qualitative subject-matter experts to inform statistical social network analysis. We show how the information from the subjective probability distributions can be used as input to Bayesian hierarchical models for network data. In the spirit of “proof of concept,” the results of a test of the technology are reported. Our findings show that human subjects can use the elicitation tool effectively, supplying attribute and edge information to update a network indicative of a covert one.

Keywords

covert networksBayesian elicitationlink prediction
Type
Research Article
Information
Network Science , Volume 1 , Issue 1 , April 2013 , pp. 68 - 94
Copyright
Copyright © Cambridge University Press 2013

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Alpert, M., & Raiffa, H. (1982). A progress report on the training of probability assessors. In Kahneman, D., Slovic, P., & Tversky, A. (Eds.). Judgment under uncertainty: Heuristics and biases (pp. 294–305). Cambridge: Cambridge University Press.Google Scholar
Batchelder, W. H. (2009). Cultural consensus theory: Aggregating expert judgments about ties in a social network. Social Computing and Behavioral Modeling, 10, 1–9.Google Scholar
Bedrick, E. J., Christensen, R., & Johnson, W. O. (1997). Bayesian binomial regression predicting survival at a trauma center. The American Statistician, 51, 211–218.Google Scholar
Belkin, N. J., Brooks, H. M., & Daniels, P. J. (1987). Knowledge elicitation using discourse analysis. International Journal of Man–Machine Studies, 27, 127–144.Google Scholar
Borgatti, S. P. (2006). Identifying sets of key players in a social network. Computational & Mathematical Organization Theory, 12, 21–34.CrossRefGoogle Scholar
Budnitz, R. J., Apostolakis, G., Boore, D. M., Cluff, L. S., Coppersmith, K. J., Cornell, C. A., & Morris, P. A. (1998). Use of technical expert panels: Applications to probabilistic seismic hazard analysis. Risk Analysis, 18, 463–469.CrossRefGoogle Scholar
Bunn, D. W. (1979). Estimation of subjective probability distributions in forecasting and decision making. Technical Forecasting and Social Change, 14, 205–216.Google Scholar
Burt, R. S. (1987). A note on missing network data in the general social survey. Social Networks, 9, 63–73.CrossRefGoogle Scholar
Carley, K. (2006). A dynamic network approach to the assessment of terrorist groups and the impact of alternative courses of action. Pre-proceedings, Visualizing Network Information IST-063/RWS-010. Royal Danish Defence College, Copenhagen.Google Scholar
Carley, K. M. (2003). Dynamic network analysis. In Breiger, R., Carley, K. & Pattison, P. (Eds.), Dynamic social network modeling and analysis (pp. 1–13). Committee on Human Factors. National Research Council. Washington, DC: The National Academies Press.Google Scholar
Carley, K. M. (2004). Estimating vulnerabilities in large covert networks using multi-level data. Proceedings of the NAACSOS Conference, Pittsburgh, PA.Google Scholar
Carlin, B. P., Chaloner, K., Church, T., Louis, T. A., & Matts, J. P. (1993). Bayesian approaches for monitoring clinical trials with an application to toxoplasti encephalitis pro-phylaxis. The Statistician, 42, 355–367.CrossRefGoogle Scholar
Carlin, B. P., Chaloner, K., Louis, T. A., & Rhames, F. S. (1995). Elicitation, monitoring, and analysis for an AIDS clinical trial. In Gatsonis, C., Hodges, J. S., Kass, R. E., & Singpurwalla, N. D. (Eds.), Case studies in Bayesian statistics (Vol. 2, pp. 48–78). New York: Springer.Google Scholar
Chai, S.-K. (1993). An organizational economics theory of antigovernment violence. Comparative Politics, 26, 99–110.Google Scholar
Chaloner, K. M., Church, T., Louis, T. A., & Matts, J. P. (1993). Graphical elicitation of a prior distribution for a clinical trail. The Statistician, 42, 341–353.CrossRefGoogle Scholar
Chaloner, K. M., & Duncan, G. T. (1983). Assessment of a beta prior distribution: PM elicitation. The Statistician, 32, 174–180.Google Scholar
Chaloner, K. M., & Duncan, G. T. (1987). Some properties of the Dirichlet-multinomial distribution and its use in prior elicitation. Communications in Statistics: Theory and Methods, 15, 511–523.Google Scholar
Clemen, R. T., & Winkler, R. L. (1999). Combining probability distributions from experts in risk analysis. Risk Analysis, 19, 187–203.Google Scholar
Cooke, R. M. (1991). Experts in uncertainty: Opinion and subjective probability in science. New York: Oxford University Press.CrossRefGoogle Scholar
Cosmides, L., & Tooby, J. (1996). Are humans good intuitive statisticians after all? Rethinking some conclusions from the literature on judgment and uncertainty. Cognition, 58, 1–73.Google Scholar
Costenbader, E., & Valente, T. W. (2003). The stability of centrality measures when networks are sampled. Social Networks, 25, 283–307.CrossRefGoogle Scholar
Crenshaw, M. (1981). The causes of terrorism. Comparative Politics, 13, 379–399.CrossRefGoogle Scholar
Ertekin, S., Hirsh, H., & Rudin, C. (2012). Learning to predict the wisdom of crowds. Paper presented at Collective Intelligence Conference 2012, Cambridge, MA.Google Scholar
Farnum, N. R., & Stanton, L. W. (1987). Some results concerning the estimation of beta distribution parameters in PERT. Journal of the Operational Research Society, 38, 287–290.Google Scholar
Fellman, P.V. and Wright, R. (2003). Modeling terrorist networks – complex systems at the mid-range, Paper prepared for the Joint Complexity Conference, London School of Economics, 16–18 September.Google Scholar
Ford, D. N., & Sterman, J. D. (1998). Expert knowledge elicitation to improve formal and mental models. System Dynamics Review, 14, 309–340.Google Scholar
Freedman, L. S., & Spiegelhalter, D. J. (1983). The assessment of subjective opinion and its use in relation to stopping rules for clinical trials. The Statistician, 32, 153–160.Google Scholar
Garthwaite, P. H., & Dickey, J. M. (1988). Quantifying expert opinion in linear regression problems. Journal of the Royal Statistical Society, Series B, 50, 462–474.Google Scholar
Garthwaite, P. H., & Dickey, J. M. (1992). Elicitation of prior distributions for variable-selection problems in regression. The Annals of Statistics, 20, 1697–1719.Google Scholar
Garthwaite, P. H., Kadane, J. B., & O'Hagan, A. (2005). Statistical methods for eliciting probability distributions. Journal of the American Statistical Association, 100, 680–700.Google Scholar
Gavasakar, U. (1988). A comparison of two elicitation methods for a prior distribution for a binomial parameter. Management Science, 34, 784–790.Google Scholar
Gelman, A., & Rubin, D. B. (1992). Inference from iterative simulation using multiple sequences. Statistical Science, 7, 457–511.CrossRefGoogle Scholar
Geweke, J. (1992). Evaluating the accuracy of sampling-based approaches to the calculation of posterior moments. In Bernardo, J. M., Smith, A. F. M., Dawid, A. P., & Berger, J. O. (Eds.), Bayesian statistics 4 (pp. 169–193). Oxford: Oxford University Press.Google Scholar
Gigerenzer, G., & Hoffrage, U. (1995). How to improve Bayesian reasoning without instruction: Frequency formats. Psychological Review, 102, 684–704.Google Scholar
Gill, J. (2008). Is partial-dimension convergence a problem for inferences from MCMC algorithms? Political Analysis, 16, 153–178.CrossRefGoogle Scholar
Gill, J., & Walker, L. D. (2005). Elicited priors for Bayesian model specifications in political science research. The Journal of Politics, 67, 841–872.Google Scholar
Goldman, L., Cook, E. F., Brand, D. A., Lee, T. H., Rouan, G. W., Weisberg, M. C., Acampora, D., Stasiulewicz, C., Walshon, J., Terranova, G., Gottlieb, L., Kobernick, M., Goldstein-Wayne, B., Copen, D., Daley, K., Brandt, A. A.Jones, D., Mellors, J., & Jakubowski, R. (1988). A computer protocol to predict myocardial infarction in emergency department patients with chest pain. New England Journal of Medicine, 318, 797–803.Google Scholar
Griffiths, T. L., & Tenenbaum, J. B. (2006). Optimal predictions in everyday cognition. Psychological Science, 17, 767–773.Google Scholar
Handcock, M. S., & Gile, K. J. (2010). Modeling social networks from sampled data. Annals of Applied Statistics, 4, 5–25.Google Scholar
Heidelberger, P., & Welch, P. D. (1981a). Adaptive spectral methods for simulation output analysis. IBM Journal of Research and Development, 25, 860–876.Google Scholar
Heidelberger, P., & Welch, P. D. (1981b). A spectral method for confidence interval generation and run length control in simulations. Communications of the Association for Computing Machinery, 24, 233–245.CrossRefGoogle Scholar
Hoff, P. D. (2005). Bilinear mixed-effects models for dyadic data. Journal of the American Statistical Association, 100, 286–295.CrossRefGoogle Scholar
Hoff, P. D. (2009). Multiplicative latent factor models for description and prediction of social networks. Computational and Mathematical Organization Theory, 15, 261–272.Google Scholar
Hoff, P. D., Raftery, A. E., & Handcock, M. S. (2002). Latent space approaches to social network analysis. Journal of the American Statistical Association, 97, 1090–1098.Google Scholar
Hoff, P. D., & Ward, M. D. (2004). Modeling dependencies in international relations networks. Political Analysis, 12, 160–175.Google Scholar
Hoffmann, S., Fischbeck, P., Krupnick, A., & Mcwilliams, M. (2007). Using expert elicitation to link foodborne illnesses in the United States to foods. Journal of Food Protection, 70, 1220–1229.CrossRefGoogle ScholarPubMed
Hogarth, R. M. (1975). Cognitive processes and the assessment of subjective probability distributions. Journal of the American Statistical Association, 70, 271–289.Google Scholar
Huisman, M., & Steglich, C. (2008). Treatment of nonresponse in longitudinal network studies. Social Networks, 30, 297–308.Google Scholar
Ibrekk, H., & Morgan, M. G. (1987). Graphical communication of uncertain quantities to non-technical people. Risk Analysis, 7, 519–529.Google Scholar
Jennison, C., & Turnbull, B. W. (1990). Statistical approaches to interim monitoring of medical trials: A review and commentary. Statistical Science, 5, 299–317.CrossRefGoogle Scholar
Johnson, V. E. (2010). Bayesian aggregation error? International Journal of Reliability and Safety, 4, 359–365.Google Scholar
Kadane, J. B. (1980). Predictive and structural methods for eliciting prior distributions. In Zellner, A. (Ed.), Bayesian analysis in econometrics and statistics (pp. 89–93). Amsterdam: North-Holland.Google Scholar
Kadane, J. B., Dickey, J. M., Winkler, R. L., Smith, W. S., & Peters, S. C. (1980). Interactive elicitation of opinion for a normal linear model. Journal of the American Statistical Association, 75, 845–854.Google Scholar
Kadane, J. B., & Winkler, R. L. (1988). Separating probability elicitation from utilities. Journal of the American Statistical Association, 83, 357–363.Google Scholar
Kadane, J. B., & Wolfson, L. J. (1998). Experiences in elicitation. Journal of the Royal Statistical Society, Series D, 47, 3–19.Google Scholar
Karabatsos, G., & Batchelder, W. H. (2003). Markov chain estimation for test theory without an answer key. Psychometrika, 68, 373–389.Google Scholar
Kass, R. E., & Greenhouse, J. B. (1989). Comments on the paper by J. H. Ware. Statistical Science, 4, 310–317.Google Scholar
Kossinets, G. (2006). Effects of missing data in social networks. Social Networks, 28, 247–268.Google Scholar
Krebs, V. E. (2002). Mapping terrorist cells. Connections, 24, 43–52.Google Scholar
Kynn, M. (2007). The ‘heuristics and biases’ bias in expert elicitation. Journal of the Royal Statistical Society, Series A, 171, 239–264.Google Scholar
Law, A. M., & Kelton, W. D. (1982). Simulation modeling and analysis. New York: McGraw-Hill.Google Scholar
Leamer, E. E. (1992). Bayesian elicitation diagnostics. Econometrica, 60, 919–942.Google Scholar
Libby, D. L., & Novick, M. R. (1982). Multivariate generalized beta distributions with applications to utility assessment. Journal of Educational Statistics, 7, 271–294.Google Scholar
Lichenstein, S., Fischoff, B., & Phillips, L. D. (1982). Calibration of probabilities: The state of the art to 1980. In Tversky, A. & Kahneman, D. (Eds.), Judgment under uncertainty: Heuristics and biases (pp. 306–334). New York: Cambridge University Press.Google Scholar
Lipkus, I. M., & Holland, J. G. (1999). The visual communication of risk. Journal of the National Cancer Institute Monographs, 25, 149–163.Google Scholar
Little, R. J. A., & Rubin, D. B. (1983). On jointly estimating parameters and missing data by maximizing the complete-data likelihood. The American Statistician, 37, 218–220.Google Scholar
Little, R. J. A., & Rubin, D. B. (2002). Statistical analysis with missing data (2nd ed.). New York: John Wiley & Sons.Google Scholar
Liu, J. S. (2001). Monte Carlo strategies in scientific computing. New York: Springer.Google Scholar
Lu, M. (2002). Enhancing project evaluation and review technique simulation through artificial neural network-based input modeling. Journal of Construction Engineering and Management, 128 (5), 438–445.Google Scholar
Merkle, E. C., & Steyvers, M. (2011). A psychological model for aggregating judgments of magnitude. In Salemo, J., Yang, S. J., Nau, D., & Chai, S.-K. (Eds.), Social computing, behavioral–cultural modeling and prediction, Lecture Notes in Computer Science Vol. 6589. Heidelberg: Springer Berlin.Google Scholar
Meyer, M. A., & Booker, J. M. (2001). Eliciting and analyzing expert judgment: A practical guide. Philadelphia, PA: SIAM Press.Google Scholar
Moon, I.-C., & Carley, K. M. (2007). Modeling and simulation of terrorist networks in social and geo-spatial dimensions. IEEE Intelligent Systems: Special Issue on Social Computing 22, 40–49.CrossRefGoogle Scholar
Morgan, M. G., Pitelka, L. F., & Shevliakova, E. (2001). Elicitation of expert judgments of climate change impacts on forest ecosystems. Climatic Change, 49, 279–307.Google Scholar
Murphy, A. H., & Winkler, R. (1974). Subjective probability forecasting experiments in meteorology: Some preliminary results. Bulletin of the American Meteorological Society, 55, 1206–1216.Google Scholar
Nowicki, K., & Snijders, T. A. B. (2001). Estimation and prediction for stochastic blockstructures. Journal of the American Statistical Association, 96, 1077–1087.Google Scholar
O'Hagan, A. (1998). Eliciting expert beliefs in substantial practical applications. The Statistician, 47 (Part 1), 21–35.Google Scholar
O'Leary, R. A., Choy, S. L., Murray, J. V., Kynn, M., Denham, R., Martin, T. G., Mengersen, K. (2009). Comparison of three expert elicitation methods for logistic regression on predicting the presence of the threatened brush-tailed rock-wallaby Petrogale penicillata. Environmetrics, 20, 379–398.CrossRefGoogle Scholar
Prelec, D. (2004). A Bayesian truth serum for subjective data. Science, 306, 462–466.Google Scholar
Risse, M. (2003). Bayesian group agents and two modes of aggregation. Synthese, 135, 347–377.Google Scholar
Robert, C., & Casella, G. (2004). Monte Carlo statistical methods (2nd ed.). New York: Springer.Google Scholar
Robins, G., Pattison, P., & Woolcock, J. (2004). Missing data in networks: Exponential random graph (p*) models for networks with non-respondents. Social Networks, 26, 257–283.Google Scholar
Rothenberg, R. (2002). From whole cloth: Making up the terrorist network. Connections, 23, 36–42.Google Scholar
Rubin, D. B. (1976). Inference and missing data. Biometrika, 63, 581–592.CrossRefGoogle Scholar
Savage, L. J. (1954). The foundations of statistics. New York: John Wiley & Sons.Google Scholar
Savage, L. J. (1971). Elicitation of personal probabilities and expectations. Journal of the American Statistical Association, 66, 783–801.Google Scholar
Schaefer, R. E., & Borcherding, K. (1973). The assessment of subjective probability distributions: A training experiment. Acta Psychologica, 37, 117–129.Google Scholar
Schafer, Joseph L. (1997). Analysis of Incomplete Multivariate Data. New York: CRC Press.Google Scholar
Sparrow, M. K. (1991). The application of network analysis to criminal intelligence: An assessment of prospects. Social Networks, 13, 251–274.Google Scholar
Spetzler, C. S., & Staël von Holstein, C.-A. S. (1975). Probability encoding and decision analysis. Management Science, 22, 340–358.Google Scholar
Tanner, M. A., & Wong, W. H. (1987) The calculation of posterior distributions by data augmentation. Journal of the American Statistical Association, 82, 528–540.CrossRefGoogle Scholar
Tetlock, P. (2006). Expert political judgment: How good is it? how can we know? Princeton, NJ: Princeton University Press.Google Scholar
Tsvetovat, M., & Carley, K. M. (2005). Structural knowledge and success of anti-terrorist activity: The downside of structural equivalence. Journal of Social Structure, 6.Google Scholar
Tsvetovat, M., & Carley, K. M. (2006). Improving effectiveness of communications sampling of covert networks. E-Social Science Conference, Manchester, UK.Google Scholar
Tversky, A., & Kahneman, D. (1974). Judgment under uncertainty: Heuristics and biases. Science, 185, 1124–1131.Google Scholar
Tversky, A. & Kahneman, D. (1982). Judgment Under Uncertainty: Heuristics and Biases. In Tversky, A. & Kahneman, D. (Eds.), Judgment Under Uncertainty: Heuristics and Biases (pp. 587–600). New York: Cambridge University Press.Google Scholar
Tversky, A., & Kahneman, D. (1983). Extensional versus intuitive reasoning: The conjunction fallacy in probability judgment. Psychological Review, 90, 293–315.Google Scholar
van Meter, K. M. (2002). Terrorists/liberators: Researching and dealing with adversarial social networks. Connections, 24, 66–78.Google Scholar
Ward, M. D., Hoff, P. D., & Lofdahl, C. L. (2003). Identifying international networks: Latent spaces and imputation. In Breiger, R., Carley, K., & Pattison, P. (Eds.), Dynamic social network modeling and analysis: Workshop summary and papers (pp. 345–359). Washington, DC: The National Academies Press.Google Scholar
Wasserman, S., & Faust, K. (1994). Social network analysis. New York: Cambridge University Press.Google Scholar
West, M. (1984). Bayesian aggregation. Journal of the Royal Statistical Society, Series A, 147, 600–607.Google Scholar
Winkler, R. (1967). The assessment of prior distributions in Bayesian analysis. Journal of the American Statistical Association, 62, 776–800.Google Scholar