Investigating conference's nature using twitter sentiment analysis

  

Investigating the nature of the conference SocmedHE21 using sentiment analysis

 

Zara Sanni 

Nuffield Placement Researcher

 

 

Introduction

SocMedHE started in 2015 with the goal of using social media to open up more higher education learning spaces online.

SocMedHE, standing for Social Media for Learning in Higher Education, is an online workshop focused on developing a social media workshop. For this investigation I am using twitter data from SocMedHE21 to investigate the nature of the conference, based in part on sentiment analysis. Sentiment analysis is also known as opinion mining, it’s a natural language processing technique used to determine whether data is positive, negative, or neutral. It’s an artificial intelligence which uses certain algorithms to categorise what a person may think about a certain topic. I will be using it on the SocMedHE21 data to see whether the conference on a whole was seen as positive or negative.

 

 

Method

The data was provided through a twitter archiving google sheet, TAGS.TAGS is a twitter scraping tool allowing you to automatically collect tweets into a google sheet from either specific people, hashtags, or search items.

 

 

 Using TAGS, #SocMedHE21 was put into the search bar. There was a follower minimum count of 0 so we could observe as much data as possible and the maximum number of tweets being 3000, between a time range of December 6^th, 2021 and January 24^th, 2022. The number of tweets using the SocMedHE hashtag was 419 and the number of unique tweets was 377, which is very close to the actual number of tweets which indicates to some sort of conversation and discussion happening between twitter users.

 

TAGSExplorer is an interactive version of the twitter conversations gathered where graphing is used to visually present the data. Using this I was able to see all the connections between the twitter users. The picture shows the names of all the users involved in the SocMedHE22 conversation; the bigger the name, the more connections and conversations the person had with other people. When you click on a person's username you can see all the tweets, they’ve made in regards to SocMedHE21 and their replies and mentions. Here you can see that the top conversationalist isn’t the official SocMedHE account that I expected, instead it was one of the conference attendees.

 

Using the TAGSExplorer arch, I was able to look at the activity of users between the dates given above and found, as expected, a lot of discourse in December, on December 14^th and the few days following but there did continue to be conversation happening some time after. There were at least a couple tweets fortnightly on twitter most of which were for a particular person or from a bot (an autonomous program on the internet or a network that can interact with systems or users) showing results from an weekly run of another social media analysis tool NodeXL. What was of interest was that the people named in the tweet were happy to retweet that tweet. I do think there is a positive correlation between twitter likes and mentions, specifically with, but not limited to, the official SocMedHE account. as most of the conference attendees would be following it any interaction with the account is more likely to be seen especially because the community involved in this conference are such conversationalists and the official account itself does also interact and retweet with the community.

 

Results

LTHEchat is a weekly learning and teaching in higher education chat where brainstorming in virtual learning spaces allows for conversations and the sharing of ideas to individuals in the education sector. Many of the LTHEchat community attended the SocMedHE conference so, using the words collected I chose the first 150 as they have all been used at least 10 times, making sure I factored in for spelling mistakes, adding misspelt words of the same nature together. Using sentiment analysis, I labelled the words either neutral positive or negative; for example, names of people have been labelled as neutral but words such as creative, change, and better positive. I didn’t find any words to label as negative which shows the nature of the conversation to be beneficial and enthusiastic of the topics discussed in the conference and what is to come of it. Of the 2992 words sampled, I marked 1152 of them as positive which counts to 38.5%. after doing this I converted the data into a CSV file and put it into wordart to create a word cloud with the most used words. I gave positive words the colour green and the neutral words I made black.

 

After using sentiment analysis to look at the data, I can conclude that the nature of the SocMedHE conference to be a positive one in which people are conversing in a productive way, carrying on conversations months after the conference which shows …

Next, to continue and further this research I think we should look at alternative software to sentiment analysis to analyse the data and see if the results are still the same. For example, Amazon Comprehend is a text analysis software that users can use to ‘find insights and relationships in text”, it identifies key phrases, people, events etc and can determine whether it is positive or negative. This is a much more thorough and in-depth way to analyse the data and will also allow us to see things we may have missed due to limiting factors.

 

 

 

 

 

Bibliography

 

https://blogs.shu.ac.uk/socmedhe/16-with-a-little-help-from-my-followers-facilitating-the-lthechat/

 

wordart.com

 

https://www.g2.com/products/sas-sentiment-analysis/competitors/alternatives

 

https://www.sciencedirect.com/science/article/pii/S2090447914000550

Recent clustering paper

 Zhou, Bing, Lu, Bei and Saeidlou, Salman 2022. A hybrid clustering method based on the several diverse basic clustering and meta-clustering aggregation technique. Cybernetics and Systems. 53 (7), pp. 1-27. https://doi.org/10.1080/01969722.2022.2110682

A hybrid clustering method based on the several diverse basic clustering and meta-clustering aggregation technique

Abstract

In hybrid clustering, several basic clustering is first generated and then for the clustering aggregation, a function is used in order to create a final clustering that is similar to all the basic clustering as much as possible. The input of this function is all basic clustering and its output is a clustering called clustering agreement. However, this claim is correct if some conditions are met. This study has provided a hybrid clustering method. This study has used the basic k-means clustering method as a basic cluster. Also, this study has increased the diversity of consensus by adopting some measures. Here, the aggregation process of the basic clusters is done by the meta-clustering technique, where the primary clusters are re-clustered to form the final clusters. The proposed hybrid clustering method has the advantages of k-means, its high speed, as well as it does not have its major weaknesses, the inability to detect non-spherical and non-uniform clusters. In the empirical studies, we have evaluated the proposed hybrid clustering method with other up-to-date and robust clustering methods on the different datasets and compared them. According to the simulation results, the proposed hybrid clustering method is stronger than other clustering methods.

Keywords Artificial intelligence; Information systems; Software; Aggregation techniques; Diversity of clustering; Hybrid clustering; Meta-clustering

Year 2022

Journal Cybernetics and Systems

Journal citation 53 (7), pp. 1-27

Publisher Taylor & Francis

ISSN 0196-9722

1087-6553

Digital Object Identifier (DOI) https://doi.org/10.1080/01969722.2022.2110682

Official URL https://www.tandfonline.com/doi/full/10.1080/01969722.2022.2110682

References

Abapour, N., A. Shafiesabet, and R. Mahboub. 2021. A novel security based routing method using ant colony optimization algorithms and RPL protocol in the IoT networks. International Journal of Electrical and Computer Sciences 3 (1):1-9.

Azimi, J., and X. Fern. 2009. Adaptive cluster ensemble selection. In Twenty-First International Joint Conference on Artificial Intelligence, Vol. 9, 992-7, California, USA, July 11-17.

Bai, L., J. Liang, and F. Cao. 2020. A multiple k-means clustering ensemble algorithm to find nonlinearly separable clusters. Information Fusion 61:36-47. . https://doi.org/10.1016/j.inffus.2020.03.009

Berahmand, K., E. Nasiri, R. Pir Mohammadiani, and Y. Li. 2021. Spectral clustering on protein-protein interaction networks via constructing affinity matrix using attributed graph embedding. Computers in Biology and Medicine 138:104933. https://doi.org/10.1016/j.compbiomed.2021.104933 PMid:34655897

Bouyer, A, and A. Hatamlou. 2018. An efficient hybrid clustering method based on improved cuckoo optimization and modified particle swarm optimization algorithms. Applied Soft Computing 67:172-82. . https://doi.org/10.1016/j.asoc.2018.03.011

Ester, M., H. P. Kriegel, J. Sander, and X. Xu. 1996. A density-based algorithm for discovering clusters in large spatial databases with noise. In KDD-96, 226-31, Portland, Oregon, USA, August 2-4.

Forouzandeh, S., K. Berahmand, E. Nasiri, and M. Rostami. 2021. A hotel recommender system for tourists using the Artificial Bee Colony Algorithm and Fuzzy TOPSIS Model: A case study of tripadvisor. International Journal of Information Technology & Decision Making 20 (1):399-429. . https://doi.org/10.1142/S0219622020500522

Fred, A. L., and A. K. Jain. 2005. Combining multiple clusterings using evidence accumulation. IEEE Transactions on Pattern Analysis and Machine Intelligence 27 (6):835-50. https://doi.org/10.1109/TPAMI.2005.113 PMid:15943417

Ghobaei-Arani, M. 2021. A workload clustering based resource provisioning mechanism using Biogeography based optimization technique in the cloud based systems. Soft Computing 25 (5):3813-30. . https://doi.org/10.1007/s00500-020-05409-2

Ghobaei-Arani, M., and A. Shahidinejad. 2021. An efficient resource provisioning approach for analyzing cloud workloads: a metaheuristic-based clustering approach. The Journal of Supercomputing 77 (1):711-50. . https://doi.org/10.1007/s11227-020-03296-w

Golalipour, K., E. Akbari, S. S. Hamidi, M. Lee, and R. Enayatifar. 2021. From clustering to clustering ensemble selection: A review. Engineering Applications of Artificial Intelligence 104:104388. . https://doi.org/10.1016/j.engappai.2021.104388

Golrou, A., A. Sheikhani, A. M. Nasrabadi, and M. R. Saebipour. 2018. Enhancement of sleep quality and stability using acoustic stimulation during slow wave sleep. International Clinical Neuroscience Journal 5 (4):126-34. . https://doi.org/10.15171/icnj.2018.25

Hamidi, S. S., E. Akbari, and H. Motameni. 2019. Consensus clustering algorithm based on the automatic partitioning similarity graph. Data & Knowledge Engineering 124:101754. . https://doi.org/10.1016/j.datak.2019.101754

Hansen, P., and N. Mladenović. 2001. J-means: A new local search heuristic for minimum sum of squares clustering. Pattern Recognition 34 (2):405-13. . https://doi.org/10.1016/S0031-3203(99)00216-2

Huang, D., C. D. Wang, and J. H. Lai. 2017. LWMC: A locally weighted meta-clustering algorithm for ensemble clustering. In International Conference on Neural Information Processing, 167-76. Cham: Springer. https://doi.org/10.1007/978-3-319-70139-4_17

Huang, D., C. D. Wang, J. S. Wu, J. H. Lai, and C. K. Kwoh. 2020. Ultra-scalable spectral clustering and ensemble clustering. IEEE Transactions on Knowledge and Data Engineering 32 (6):1212-26. . https://doi.org/10.1109/TKDE.2019.2903410

Iam-On, N., T. Boongoen, S. Garrett, and C. Price. 2011. A link-based approach to the cluster ensemble problem. IEEE Transactions on Pattern Analysis and Machine Intelligence 33 (12):2396-409. . https://doi.org/10.1109/TPAMI.2011.84 PMid:21576752

Jadidi, A., and M. R. Dizadji. 2021. Node clustering in binary asymmetric stochastic block model with noisy label attributes via SDP. In 2021 International Conference on Smart Applications, Communications and Networking (SmartNets), 1-6. New York: IEEE. . https://doi.org/10.1109/SmartNets50376.2021.9555421

Jain, A. K. 2010. Data clustering: 50 years beyond k-means. Pattern Recognition Letters 31 (8):651-66. . https://doi.org/10.1016/j.patrec.2009.09.011

Jiang, H., S. Yi, J. Li, F. Yang, and X. Hu. 2010. Ant clustering algorithm with K-harmonic means clustering. Expert Systems with Applications 37 (12):8679-84. . https://doi.org/10.1016/j.eswa.2010.06.061

Khedairia, S., and M. T. Khadir. 2022. A multiple clustering combination approach based on iterative voting process. Journal of King Saud University - Computer and Information Sciences 34 (1):1370-80. . https://doi.org/10.1016/j.jksuci.2019.09.013

Li, F., Y. Qian, and J. Wang. 2021. GoT: A growing tree model for clustering ensemble. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35, 8349-56, California, USA, February 2-9.

Li, T., A. Rezaeipanah, and E. M. T. El Din. 2022. An ensemble agglomerative hierarchical clustering algorithm based on clusters clustering technique and the novel similarity measurement. Journal of King Saud University - Computer and Information Sciences 34 (6):3828-42. . https://doi.org/10.1016/j.jksuci.2022.04.010

Ma, T., Z. Zhang, L. Guo, X. Wang, Y. Qian, and N. Al-Nabhan. 2021. Semi-supervised Selective Clustering Ensemble based on constraint information. Neurocomputing 462:412-25. . https://doi.org/10.1016/j.neucom.2021.07.056

Mojarad, M., F. Sarhangnia, A. Rezaeipanah, H. Parvin, and S. Nejatian. 2021. Modeling hereditary disease behavior using an innovative similarity criterion and ensemble clustering. Current Bioinformatics 16 (5):749-64. . https://doi.org/10.2174/1574893616999210128175715

Movahhed Neya, N., S. Saberi, and B. Rezaie. 2022. Design of an adaptive controller to capture maximum power from a variable speed wind turbine system without any prior knowledge of system parameters. Transactions of the Institute of Measurement and Control 44 (3):609-19. . https://doi.org/10.1177/01423312211039041

Nasiri, E., K. Berahmand, Z. Samei, and Y. Li. 2022. Impact of centrality measures on the common neighbors in link prediction for multiplex networks. Big Data 10 (2):138-50. https://doi.org/10.1089/big.2021.0254 PMid:35333606

Ng, A., M. Jordan, and Y. Weiss. 2001. On spectral clustering: Analysis and an algorithm. Advances in Neural Information Processing Systems 14:849-56.

Nguyen, N., and R. Caruana. 2007. Consensus clusterings. In Seventh IEEE International Conference on Data Mining (ICDM 2007), 607-12. New York: IEEE. . https://doi.org/10.1109/ICDM.2007.73

Niu, H., N. Khozouie, H. Parvin, H. Alinejad-Rokny, A. Beheshti, and M. R. Mahmoudi. 2020. An ensemble of locally reliable cluster solutions. Applied Sciences 10 (5):1891. . https://doi.org/10.3390/app10051891

Rezaeipanah, A., P. Amiri, H. Nazari, M. Mojarad, and H. Parvin. 2021. An energy-aware hybrid approach for wireless sensor networks using re-clustering-based multi-hop routing. Wireless Personal Communications 120 (4):3293-314. . https://doi.org/10.1007/s11277-021-08614-w

Rezaeipanah, A., H. Nazari, and G. Ahmadi. 2019. A hybrid approach for prolonging lifetime of wireless sensor networks using genetic algorithm and online clustering. Journal of Computing Science and Engineering 13 (4):163-74. . https://doi.org/10.5626/JCSE.2019.13.4.163

Rodriguez, A., and A. Laio. 2014. Clustering by fast search and find of density peaks. Science (New York, N.Y.) 344 (6191):1492-6. . https://doi.org/10.1126/science.1242072 PMid:24970081

Shahidinejad, A., M. Ghobaei-Arani, and L. Esmaeili. 2020. An elastic controller using Colored Petri Nets in cloud computing environment. Cluster Computing 23 (2):1045-71. . https://doi.org/10.1007/s10586-019-02972-8

Strehl, A., and J. Ghosh. 2002. Cluster ensembles - A knowledge reuse framework for combining multiple partitions. Journal of Machine Learning Research 3:583-617.

Sun, S., S. Wang, G. Zhang, and J. Zheng. 2018. A decomposition-clustering-ensemble learning approach for solar radiation forecasting. Solar Energy 163:189-99. . https://doi.org/10.1016/j.solener.2018.02.006

Tan, H., Y. Tian, L. Wang, and G. Lin. 2020. Name disambiguation using meta clusters and clustering ensemble. Journal of Intelligent & Fuzzy Systems 38 (2):1559-68. . https://doi.org/10.3233/JIFS-179519

Topchy, A., A. K. Jain, and W. Punch. 2005. Clustering ensembles: Models of consensus and weak partitions. IEEE Transactions on Pattern Analysis and Machine Intelligence 27 (12):1866-81. . https://doi.org/10.1109/TPAMI.2005.237 PMid:16355656

Trik, M., A. M. N. G. Molk, F. Ghasemi, and P. Pouryeganeh. 2022. A hybrid selection strategy based on traffic analysis for improving performance in networks on chip. Journal of Sensors 2022:1-19. . https://doi.org/10.1155/2022/3112170

Trik, M., S. Pour Mozaffari, and A. M. Bidgoli. 2021. Providing an adaptive routing along with a hybrid selection strategy to increase efficiency in NoC-based neuromorphic systems. Computational Intelligence and Neuroscience 2021:8338903. . https://doi.org/10.1155/2021/8338903 PMid:34567105 PMCid:PMC8460363

Walid, W., M. Awais, A. Ahmed, G. Masera, and M. Martina. 2021. Real-time implementation of fast discriminative scale space tracking algorithm. Journal of Real-Time Image Processing 18 (6):2347-60. . https://doi.org/10.1007/s11554-021-01119-6

Wei, S., Z. Li, and C. Zhang. 2018. Combined constraint-based with metric-based in semi-supervised clustering ensemble. International Journal of Machine Learning and Cybernetics 9 (7):1085-100. . https://doi.org/10.1007/s13042-016-0628-6

Wei, Y., S. Sun, J. Ma, S. Wang, and K. K. Lai. 2019. A decomposition clustering ensemble learning approach for forecasting foreign exchange rates. Journal of Management Science and Engineering 4 (1):45-54. https://doi.org/10.1016/j.jmse.2019.02.001

Yang, W., Y. Zhang, H. Wang, P. Deng, and T. Li. 2021. Hybrid genetic model for clustering ensemble. Knowledge-Based Systems 231:107457. . https://doi.org/10.1016/j.knosys.2021.107457

Zhang, B., M. Hsu, and U. Dayal. 2000. K-harmonic means-a spatial clustering algorithm with boosting. In International Workshop on Temporal, Spatial, and Spatio-Temporal Data Mining, 31-45. Berlin, Heidelberg: Springer. https://doi.org/10.1007/3-540-45244-3_4

Zhao, Q., Y. Zhu, D. Wan, Y. Yu, and Y. Lu. 2019. Similarity analysis of small-and medium-sized watersheds based on clustering ensemble model. Water 12 (1):69. . https://doi.org/10.3390/w12010069

Zheng, Y., Z. Long, C. Wei, and H. Wang. 2021. Particle swarm optimization for clustering ensemble. In 2021 16th International Conference on Intelligent Systems and Knowledge Engineering (ISKE), 385-91. New York: IEEE. . https://doi.org/10.1109/ISKE54062.2021.9755338

Zhou, Z. H., and W. Tang. 2006. Clusterer ensemble. Knowledge-Based Systems 19 (1):77-83. . https://doi.org/10.1016/j.knosys.2005.11.003

Zhu, X., B. Fei, D. Liu, and W. Bao. 2021. Adaptive clustering ensemble method based on uncertain entropy decision-making. In 2021 IEEE 20th International Conference on Trust, Security and Privacy in Computing and Communications (TrustCom), 61-7. New York: IEEE. https://doi.org/10.1109/TrustCom53373.2021.00026

Analysis of expertise in intelligence analysis for human-centered AI

Recently published paper 

S. Hepenstal, L. Zhang and B. L. William Wong, "An analysis of expertise in intelligence analysis to support the design of Human-Centered Artificial Intelligence," 2021 IEEE International Conference on Systems, Man, and Cybernetics (SMC), 2021, pp. 107-112, doi: 10.1109/SMC52423.2021.9659095.

ttps://doi.org/10.1109/SMC52423.2021.9659095 Abstract Intelligence analysis involves unpredictable processes and decision making about complex domains where analysts rely upon expertise. Artificial Intelligence (AI) systems could support analysts as they perform analysis tasks, to enhance their expertise. However, systems must also be cognisant about how expertise is gained and designed so that this is not impinged. In this paper, we describe the results of Cognitive Task Analysis interviews with 6 experienced intelligence analysts. We capture themes, in terms of their decision making paths during an analysis task, and highlight how each theme is both influenced by expertise and an influence upon expertise. We also identify important interdependencies between themes. We propose that our findings can be used to help design Human-Centered AI (HCAI) systems for supporting intelligence analysts. To read more go to: https://doi.org/10.1109/SMC52423.2021.9659095 References 1.   G. A. Klein, R. Calderwood and D. MacGregor, "Critical decision method for eliciting knowledge", IEEE Transactions on Systems Man and Cybernetics, vol. 19, no. 3, pp. 462-472, 1989. https://doi.org/10.1109/21.31053   2.   B. L. W. Wong, Critical Decision Method Data Analysis, Lawrence Erlbaun Associates, pp. 327-346, 1993.     3.   M. Gerber, B. W. Wong and N. Kodagoda, "How analysts think: Intuition leap of faith and insight", Proceedings of the Human Factors and Ergonomics Society Annual Meeting, vol. 60, no. 1, pp. 173-177, 2016, [online] Available: https://doi.org/10.1177/1541931213601039. https://doi.org/10.1177/1541931213601039     4.   "National intelligence machinery", [online] Available: https://www.gov.uk/government/publications/national-intelligence-machinery.     5.   B. W. Wong and N. Kodagoda, "How analysts think: Inference making strategies", Proceedings of the Human Factors and Ergonomics Society Annual Meeting, vol. 59, no. 1, pp. 269-273, 2015, [online] Available: https://doi.org/10.1177/1541931215591055. https://doi.org/10.1177/1541931215591055     6.   G. Tecuci, D. Schum, M. Boicu, D. Marcu and B. Hamilton, "Intelligence analysis as agent-assisted discovery of evidence hypotheses and arguments" in Advances in Intelligent Decision Technologies, Berlin, Heidelberg:Springer Berlin Heidelberg, pp. 1-10, 2010. https://doi.org/10.1007/978-3-642-14616-9_1     7.   J. D. Corkill, T. K. Cunow, E. Ashton and A. East, "Attributes of an analyst: What we can learn from the intelligence analysts job description", 8th Australian Security and Intelligence Conference, 2015.     8.   N. Quarmby and L. J. Young, "Managing intelligence: The art of influence" in , Sydney:The Federation Press, 2010.     9.   D. M. Allen, "Building a better strategic analyst: a critical review of the U.S. army's all source analyst training program" in , Fort Leavenworth:United States Army Command and General Staff College, pp. 59, 2008.     10.   G. Klein, A recognition-primed decision (RPD) model of rapid decision making, Abex Publishing, pp. 138-147, 1993.   11.   J. Rasmussen, "Taxonomy for cognitive work analysis", [online] Available: .     12.   D. Leslie, "Understanding artificial intelligence ethics and safety: A guide for the responsible design and implementation of ai systems in the public sector", 2019, [online] Available: https://doi.org/10.5281/zenodo.3240529. https://doi.org/10.2139/ssrn.3403301   13.   S. Hepenstal, N. Kodagoda, L. Zhang, P. Paudyal and B. L. W. Wong, "Algorithmic transparency of conversational agents", Joint Proceedings of the ACM IUI 2019 Workshops co-located with the 24th ACM Conference on Intelligent User Interfaces (ACM IUI 2019), vol. 2327, March 20, 2019, [online] Available: http://ceur-ws.org/Vol-2327/IUI19WS-IUIATEC-1.pdf.     14.   W. Xu, "Toward human-centered ai: A perspective from human-computer interaction", Interactions, vol. 26, no. 4, pp. 42-46, Jun. 2019, [online] Available: https://doi.org/10.1145/3328485. https://doi.org/10.1145/3328485   15.   B. L. W. Wong and A. Blandford, "Analysing ambulance dispatcher decision making: trialing emergent themes analysis" in , Canberra, Australia:Ergonomics Society of Australia, 2002.   16.   B. L. W. Wong and A. Blandford, "Describing situation awareness at an emergency medical dispatch centre", Proceedings of the Human Factors and Ergonomics Society Annual Meeting, vol. 48, pp. 285-289, 2004. https://doi.org/10.1177/154193120404800302     17.   M. R. Endsley, "Toward a theory of situation awareness in dynamic systems", Human Factors, vol. 37, no. 1, pp. 32-64, 1995, [online] Available: https://doi.org/10.1518/001872095779049543. https://doi.org/10.1518/001872095779049543   18.   G. Klein, Seeing what others don't. The Remarkable Ways We Gain Insights, Nicholas Brealey Publishing, 2017.   19.   F. Qiao, P. Li, X. Zhang, Z. Ding, J. Cheng and H. Wang, "Predicting social unrest events with hidden markov models using gdelt", Discrete Dynamics in Nature and Society, vol. 2017, pp. 1-13, 2017. https://doi.org/10.1155/2017/8180272   References & Cited By Select All 1. G. A. Klein, R. Calderwood and D. MacGregor, "Critical decision method for eliciting knowledge", IEEE Transactions on Systems Man and Cybernetics, vol. 19, no. 3, pp. 462-472, 1989. Show in Context View Article   Google Scholar  2. B. L. W. Wong, Critical Decision Method Data Analysis, Lawrence Erlbaun Associates, pp. 327-346, 1993. Show in Context Google Scholar  3. M. Gerber, B. W. Wong and N. Kodagoda, "How analysts think: Intuition leap of faith and insight", Proceedings of the Human Factors and Ergonomics Society Annual Meeting, vol. 60, no. 1, pp. 173-177, 2016, [online] Available: https://doi.org/10.1177/1541931213601039. Show in Context CrossRef  Google Scholar  4. "National intelligence machinery", [online] Available: https://www.gov.uk/government/publications/national-intelligence-machinery. Show in Context Google Scholar  5. B. W. Wong and N. Kodagoda, "How analysts think: Inference making strategies", Proceedings of the Human Factors and Ergonomics Society Annual Meeting, vol. 59, no. 1, pp. 269-273, 2015, [online] Available: https://doi.org/10.1177/1541931215591055. Show in Context CrossRef  Google Scholar  6. G. Tecuci, D. Schum, M. Boicu, D. Marcu and B. Hamilton, "Intelligence analysis as agent-assisted discovery of evidence hypotheses and arguments" in Advances in Intelligent Decision Technologies, Berlin, Heidelberg:Springer Berlin Heidelberg, pp. 1-10, 2010. Show in Context CrossRef  Google Scholar  7. J. D. Corkill, T. K. Cunow, E. Ashton and A. East, "Attributes of an analyst: What we can learn from the intelligence analysts job description", 8th Australian Security and Intelligence Conference, 2015. Show in Context Google Scholar  8. N. Quarmby and L. J. Young, "Managing intelligence: The art of influence" in , Sydney:The Federation Press, 2010. Show in Context Google Scholar  9. D. M. Allen, "Building a better strategic analyst: a critical review of the U.S. army’s all source analyst training program" in , Fort Leavenworth:United States Army Command and General Staff College, pp. 59, 2008. Show in Context Google Scholar  10. G. Klein, A recognition-primed decision (RPD) model of rapid decision making, Abex Publishing, pp. 138-147, 1993. Show in Context Google Scholar  11. J. Rasmussen, "Taxonomy for cognitive work analysis", [online] Available: . Show in Context View Article   Google Scholar  12. D. Leslie, "Understanding artificial intelligence ethics and safety: A guide for the responsible design and implementation of ai systems in the public sector", 2019, [online] Available: https://doi.org/10.5281/zenodo.3240529. Show in Context CrossRef  Google Scholar  13. S. Hepenstal, N. Kodagoda, L. Zhang, P. Paudyal and B. L. W. Wong, "Algorithmic transparency of conversational agents", Joint Proceedings of the ACM IUI 2019 Workshops co-located with the 24th ACM Conference on Intelligent User Interfaces (ACM IUI 2019), vol. 2327, March 20, 2019, [online] Available: http://ceur-ws.org/Vol-2327/IUI19WS-IUIATEC-1.pdf. Show in Context Google Scholar  14. W. Xu, "Toward human-centered ai: A perspective from human-computer interaction", Interactions, vol. 26, no. 4, pp. 42-46, Jun. 2019, [online] Available: https://doi.org/10.1145/3328485. Show in Context Google Scholar  15. B. L. W. Wong and A. Blandford, "Analysing ambulance dispatcher decision making: trialing emergent themes analysis" in , Canberra, Australia:Ergonomics Society of Australia, 2002. Show in Context Google Scholar  16. B. L. W. Wong and A. Blandford, "Describing situation awareness at an emergency medical dispatch centre", Proceedings of the Human Factors and Ergonomics Society Annual Meeting, vol. 48, pp. 285-289, 2004. Show in Context CrossRef  Google Scholar  17. M. R. Endsley, "Toward a theory of situation awareness in dynamic systems", Human Factors, vol. 37, no. 1, pp. 32-64, 1995, [online] Available: https://doi.org/10.1518/001872095779049543. Show in Context CrossRef  Google Scholar  18. G. Klein, Seeing what others don’t. The Remarkable Ways We Gain Insights, Nicholas Brealey Publishing, 2017. Show in Context Google Scholar  19. F. Qiao, P. Li, X. Zhang, Z. Ding, J. Cheng and H. Wang, "Predicting social unrest events with hidden markov models using gdelt", Discrete Dynamics in Nature and Society, vol. 2017, pp. 1-13, 2017. CrossRef  Google Scholar