Joint Workshop of the 5th Extraction and Evaluation of Knowledge Entities from Scientific Documents (EEKE2024) and the 4th AI + Informetrics (AII2024)

at the iConference2024, Changchun, China and Online


News: Prof. Karin Verspoor (RMIT University) has confirmed our invitation for a keynote in EEKE - AII 2024 and titile of the keynote is: Opportunities for AI-enabled scientific knowledge exploration, analysis, and discovery.

News: : Since EEKE-AII Workshop is hosted by iConference2024, at least one author per paper must register, see instructions here <>.

Call for Papers

You are invited to participate in the Joint Workshop of the 5th Extraction and Evaluation of Knowledge Entities from Scientific Documents (EEKE2024) and the 4th AI + Informetrics (AII2024), to be held as part of the iConference2024, Changchun, China and Online, April 22 - 26, 2024

Aim of the Workshop

In the era of big data, massive amounts of information and data have dramatically changed human civilization. The broad availability of information provides more opportunities for people, but a new challenge is rising: how can we obtain useful knowledge from numerous information sources. A knowledge entity is a relatively independent and integral knowledge module in a special discipline or a research domain [1]. As a crucial medium for knowledge transmission, scientific documents that contain a large number of knowledge entities attract the attention of scholars [2]. Complementarily, informetrics, known as the study of quantitative aspects of information, has gained great benefits from artificial intelligence (AI), with its capacities in analyzing unstructured scalable data and streams, understanding uncertain semantics, and developing robust and repeatable models. Incorporating informetrics with AI techniques has demonstrated enormous success in turning big data into big value and impact. For example, deep learning approaches enlighten studies of pattern recognition and further leverage time series to track technological change. However, how to effectively cohere the power of AI and informetrics to create cross-disciplinary solutions is still elusive from neither theoretical nor practical perspectives.

This workshop aims to engage related communities in open problems in the extraction and evaluation of knowledge entities from scientific documents and AI + Informetrics. Specifically, knowledge entities in scientific documents may include method entities, tasks, dataset and metrics, software and tools, etc [3]. Knowledge entity application includes the construction of a knowledge entity graph and roadmap, modeling functions of knowledge entity citations, etc. There are some online platforms based on knowledge entities, e.g., SAGE Research Methods and ‘SOTA’ project. In parallel, this workshop also targets certain unsolved issues in AI + Informetrics and a wide range of its practical scenarios including: Cohering AI and informetrics to fulfill cross-disciplinary gaps from either theoretical or practical perspectives; elaborating AI-empowered informetric models with enhanced capabilities in robustness, adaptability, and effectiveness, and leveraging knowledge, concepts, and models in information management to strengthen the interpretability of AI + Informetrics to adapt to empirical needs in real-world cases [4].
This joint workshop entitles these two cutting-edge and cross-disciplinary directions as:

  • Extraction and Evaluation of Knowledge Entity (EEKE), highlighting the development of intelligent methods for identifying knowledge entities from scientific documents, and promoting their application in broad information studies.
  • AI + Informetrics (AII), emphasizing endeavors in interacting AI and informetrics by constructing fundamental theories, developing novel methodologies, bridging conceptual knowledge with practical uses, and creating real-word solutions.
This workshop is to gather researchers and practical users to open a collaborative platform for exchanging ideas, sharing pilot studies, and scoping future directions on this cutting-edge venue.


Workshop Topics

This workshop is primarily designed for academic researchers in broad information and library sciences, science of science, artificial intelligence, and will also be of interest to librarians, ST&I administrators and policymakers, and practitioners in any related sectors.
We invite stimulating research on topics including, but not limited to, methods of knowledge entity extraction and applications of knowledge entity. Specific examples of fields of interest include:

  • Task and methodology from scientific documents
  • Model and algorithmize entity extraction from scientific documents
  • Dataset and metrics mention extraction from scientific documents
  • Software and tool extraction from scientific documents
  • Knowledge entity summarization
  • Relation extraction of knowledge entity
  • Modeling function of knowledge entity citation
  • Informetrics with machine learning (including deep learning)
  • Informetrics with natural language processing or computational linguistics
  • Informetrics with computer vision
  • Informetrics with other related AI techniques (e.g., information retrieval)
  • AI for science of science
  • AI for science, technology, & innovation
  • AI for research policy and strategic management
  • Application of knowledge entity extraction
  • Applications of AI-empowered informetrics



Keynote:  Opportunities for AI-enabled scientific knowledge exploration, analysis, and discovery

Abstract: A wealth of information relevant to biomedicine exists in the textual resources of scientific literature, clinical notes or reports, and even patents. As largely natural language data intended for human communication rather than structured data representation, it can be challenging to find and use. However, tasks ranging from systematic reviews to protein function prediction to hypothesis generation can benefit from organisation, mining, and modelling of these resources. In this talk, I will describe a body of work based on developing artificial intelligence and natural language processing methods for structuring of key information described in textual resources, and the use of these methods in a range of biomedical and biochemical applications.

Professor Karin Verspoor is Executive Dean of the School of Computing Technologies at RMIT University in Melbourne, Australia. She is a Fellow of the Australasian Institute of Digital Health, a 2021 “Brilliant Woman in Digital Health”, and was selected as a finalist in the Women in AI Australia/New Zealand Awards 2022 for “AI in Innovation”. Karin is passionate about using artificial intelligence to enable biological discovery and clinical decision support from data. Her work has a specific emphasis on the use of natural language processing to transform unstructured data in biomedicine into actionable information. Karin held previous posts as Director of Health Technologies and Deputy Head of the School of Computing and Information Systems at the University of Melbourne, as the Scientific Director of Health and Life Sciences at NICTA Victoria Research Laboratory, at the University of Colorado School of Medicine, and at Los Alamos National Laboratory. She also spent 5 years in tech start-ups during the US Tech bubble, where she helped design an early artificial intelligence system. Karin received a BA with a double major in Computer Science and Cognitive Sciences from Rice University in Houston, TX, USA, and completed both a MSc and PhD in Cognitive Science and Natural Language at the University of Edinburgh, UK.


The workshop will be held on April 23~24, 2024 (Beijing Time), and specific activities include keynote, paper presentations and a power talk session.

April 23, 2:00pm -5:30pm (Beijing Time), Location: Room 2 (3F)
Beijing Time Content Speaker(*) Session Chair
2:00pm-2:05pm, April 23 Openning Remarks  (Online) Co-Chairs of EEKE-AII2023 (Chengzhi Zhang, Yi Zhang, Philipp Mayr, Wei Lu, Arho Suominen, Haihua Chen, and Ying Ding)
2:05pm-3:30pm, April 23 Session 1: Technology Mining
2:05-2:25 Technological Forecasting Based on Spectral Clustering for Word Frequency Time Series Han Huang*, Xiaoguang Wang and Hongyu Wang Chair:Zhinan Wang
2:25-2:45 Automated Identification of Emerging Technologies: Open Data Approach   Ljiljana Dolamic, Julian Jang-Jaccard*,  Alain Mermoud and Vincent Lenders
2:45-3:00 Technology Convergence Prediction From a Timeliness Perspective: An Improved Contribution Index in a Dynamic Network Jinzhu Zhang and Bing Yan*
3:00-3:15 A research topic evolution prediction approach based on multiplex-graph representation learning Yang Zheng, Kaiwen Shi, Yuhang Dong, Xiaoguang Wang and Hongyu Wang*
3:15-3:30 Unveiling the secret of information rediffusion process on social media from information coupling perspective: a hybrid approach of machine learning and regression model Zhen Yan*, Rong Du and Hua Wang
Coffee Break
4:00pm-5:25pm, April 23 Session 2: Entity & Relation Extraction
4:00-4:20 Biomedical Relation Extraction via Domain Knowledge and Prompt Learning Jianyuan Yuan*, Wei Du, Xiaoxia Liu and Yijia Zhang Chair: Yingyi Zhang
4:20-4:40 Identifying scientific problems and solutions: Semantic network analytics and deep learning Lu Huang, Xiaoli Cao*, Hang Ren, Chunze Zhang and Zhenxin Wu
4:40-4"55 Material performance evolution discovery based on entity extraction and social circle theory Jinzhu Zhang and Wenwen Sun*
4:55-5:10 Revealing the Country Preference on Research Method in the Field of Digital Humanities: From the Perspective of Library and Information Science Chengxi Yan* and Zhichao Fang
5:10-5:25 LLM-Resilient Bibliometrics: Factual Consistency Through Entity Triplet Extraction Alexander Sternfeld*, Andrei Kucharavy, Dimitri Percia David, Julian Jang-Jaccard and Alain Mermoud
April 24, 2:00pm -5:30pm (Beijing Time), Location: Room 2 (3F)
2:00pm-2:45pm, April 24 Keynote : Opportunities for AI-enabled scientific knowledge exploration, analysis, and discovery Karin Verspoor Chair: Yi Zhang
2:45pm-3:30pm, April 24 Session 3: Power Talk
2:45-2:50 How to Measure Information Cocoon in Academic Environment Jia Yuan, Guoxiu He and Yunhan Yang* Chair: Meijun Liu
2:50-2:55 May Generative AI Be a Reviewer on an Academic Paper? Haichen Zhou*, Xiaorong Huang, Hongjun Pu and Qi Zhang
2:55-3:00 Research on the Identification of breakthrough technology combinations driven by science Dan Wang*, Xiao Zhou, Pengwei Zhao, Juan Pang and Qiaoyang Ren
3:00-3:05 Connector and Provincial Hub Dichotomy in Scientific Collaborations Identified by Reinforcement Learning Algorithm Feifan Liu*, Shuang Zhang and Haoxiang Xia
3:05-3:10 Research on Named Entity Recognition from Patent Texts with Local Larg Language Models Chi Yu, Liang Chen* and Haiyun Xu
3:10-3:15 IRUGCN: A Graph Convolutional Network Rumor Detection Model Incorporating User Behavior Shu Zhou, Hao Wang, Zhengda Zhou*, Haohan Yi and Bin Shi
3:15-3:20 Identification of core technological topics in the new energy vehicle industry: The SAO-BERTopic topic modeling approach based on patent text mining  Jianxin Zhu,Yutong Chuang*, Zhinan Wang and Yunke Li
3:20-3:25 Research on Fine-grained S&T Entity Identification with Contextual Semantics in Think-Tank Text Mengge Sun*, Yanpeng Wang and Yang Zhao
3:25-3:30 Biomedical association inference on pandemic knowledge graphs: A comparative study Mengjia Wu*, Chao Yu, Jian Xu, Ying Ding and Yi Zhang.
Coffee Break
4:00pm-5:35pm, April 24 Session 4: AI for informetric
4:00-4:15 Understanding Citation Mobility in the Knowledge Space Shuang Zhang*, Feifan Liu and Haoxiang Xia Chair:  Jin Mao
4:15-4:30 Relationship between Team Diversity and Innovation Performance in Interdisciplinary Research Teams within the Field of Artificial Intelligence: Decision Tree Analysis Junwan Liu*, Chenchen Huang and Shuo Xu
4:30-4:45 Understanding Partnership in Scientific Collaborations A Preliminary Study from the Paper-level Perspective Chao Lu*, Mengting Li and Chenyu Zhou
4:45-5:00 Quantifying scientific novelty of doctoral theses with Bio-BERT model Alex Yang, Yi Bu, Ying Ding and Meijun Liu*
5:00-5:15 Are Disruptive Patents Less Likely to be Granted? Analyzing Scientific Gatekeeping with USPTO Patent Data (2004-2018) Lihan Yan*, Haochuan Cui and Cheng-Jun Wang
5:15-5:30 Open-mentorship team benefit disruptive ideas Bili Zheng*, Wenjing Li and Jianhua Hou
5:30-5:35 Greeting Notes of EEKE2022 Co-Chairs of EEKE-AII2023 (Chengzhi Zhang, Yi Zhang, Philipp Mayr, Wei Lu, Arho Suominen, Haihua Chen, and Ying Ding)
5::35 End of workshop


Submission Information

All submissions must be written in English, following the CEUR-ART style and should be submitted as PDF files to EasyChair.

  • Regular papers:  10 pages for full papers and 4 pages for short papers exclusive of unlimited pages for references.
  • Poster & demonstration: We welcome submissions detailing original, early findings, works in progress and industrial applications of knowledge entities extraction ande evaluation for a special poster session, possibly with a 2-minute presentation in the main session. Some research track papers will also be invited to the poster track instead, although there will be no difference in the final proceedings between poster and research track submissions. These papers should follow the same format as the research track papers but can be shorter (2 pages for poster and demo papers).

Submit a paper

All submissions will be reviewed by at least two independent reviewers. Please be aware of the fact that at least one author per paper needs to register for the workshop and attend the workshop to present the work. In case of no-show the paper (even if accepted) will be deleted from the proceedings and from the program.

Workshop proceedings will be deposited online in the CEUR workshop proceedings publication service. This way the proceedings will be permanently available and citable (digital persistent identifiers and long term preservation).

Special Issue

Accepted submissions will be invited to submit to our special issue in Technological Forecasting and Social Change. More detailed information about this special issue can be visited at:


Important Dates

All dates are Anywhere on Earth (AoE).

Deadline for submission: February 29, 2024March, 7, 2024
Notification of acceptance: March 30, 2024
Camera ready: April 10, 2024
Workshop: April 23~24, 2024


Main Organising Committee

Chengzhi Zhang ( is a professor of Department of Information Management, Nanjing University of Science and Technology, China. He received his PhD degree of Information Science from Nanjing University, China. He has published more than 100 publications, including JASIST, IPM, LISR, TFSC, Aslib JIM, JOI, OIR, SCIM, ACL, NAACL, etc. His current research interests include scientific text mining, knowledge entity extraction and evaluation, social media mining. He serves as Editorial Board Member and Managing Guest Editor for 10 international journals (Patterns, IPM, SCIM, OIR, Aslib JIM, TEL, JDIS, DIM, DI, etc.) and PC members of several international conferences in fields of natural language process and scientometrics. (

Yi Zhang ( works as a senior lecturer at the Australian Artificial Intelligence Institute, University of Technology Sydney. He holds dual Ph.D. degrees in Management Science & Engineering and in Software Engineering. His research interests align with intelligent bibliometrics - incorporating artificial intelligence and data science techniques with bibliometric indicators for broad science, technology & innovation studies. He is the recipient of the 2019 Discovery Early Career Researcher Award granted by the Australian Research Council. He serves as the Associate Editor for Technol. Forecast. & Soc. Change, the Editorial Board Member for the IEEE Trans. Eng. Manage., and the Advisory Board Member for the International Center for the Study of Research. (

Philipp Mayr ( is a team leader at the GESIS - Leibniz-Institute for the Social Sciences department Knowledge Technologies for the Social Sciences (WTS). He received his PhD in applied informetrics and information retrieval from the Berlin School of Library and Information Science at Humboldt University Berlin. He has published in top conferences and prestigious journals in the areas informetrics, information retrieval and digital libraries. His research group focuses on methods and techniques for interactive information retrieval and data set search. He was the main organizer of the BIR workshops at ECIR 2014-2021 and the BIRNDL workshops at JCDL 2016 and SIGIR 2017-2019. (

Wei Lu ( is a professor of School of Information Management and director of Information Retrieval and Knowledge Mining Center, Wuhan University. He received his PhD degree of Information Science from Wuhan University, China. His current research interests include information retrieval, text mining, QA etc. He has papers published on SIGIR, Information Sciences, JASIT, Journal of Information Science etc. He serves as diverse roles (e.g., Associate Editor, Editorial Board Member, and Managing Guest Editor) for several journals. (

Arho Suominen ( is principal scientist at the VTT Technical Research Centre of Finland and Industrial professor at Tampere University (Finland). Dr. Suominen’s research focuses on qualitative and quantitative assessment of innovation systems with a special focus on quantitative methods. His prior research has been funded by the European Commission via H2020, Academy of Finland, Finnish Funding Agency for Technology, Turku University Foundation and the Fulbright Center Finland. Through the Fulbright program, he worked as Visiting Scholar at the School of Public Policy at the Georgia Institute of Technology. Dr. Suominen has a Doctor of Science (Tech.) degree from the University of Turku and holds an Officers basic degree from the National Defence University of Finland. (

Haihua Chen ( an assistant professor in the Departmentof Information Science at the University of North Texas. He has expertise in applied data science, natural language processing, information retrieval, and text mining. He co-authored more than 40 publications in academic venues in both information science and computer science. He is serving as co-editor for The Electronic Library, the guest editor of Information Discovery & Delivery and Frontiers in Big Data special issues, and the reviewer for 14 peer reviewed journals and several international conferences. (

Ying Ding ( Bill & Lewis Suit professor at School of Information, University of Texas at Austin. She has been involved in various NIH, NSF and European-Union funded projects. She has published 240+ papers in journals, conferences, and workshops, and served as the program committee member for 200+ international conferences. She is the co-editor of book series called Semantic Web Synthesis by Morgan & Claypool publisher, the co-editor-in-chief for Data Intelligence published by MIT Press and Chinese Academy of Sciences, and serves as the editorial board member for several top journals in Information Science and Semantic Web. Her current research interests include data-driven science of science, AI in healthcare, Semantic Web, knowledge graph, data science, scholarly communication, and the application of Web technologies. (


Programme Committee

  • Alireza Abbasi, University of New South Wales (Canberra)
  • Iana Atanassova, CRIT, Université de Bourgogne Franche-Comté
  • Marc Bertin, Université Claude Bernard Lyon 1
  • Katarina Boland, GESIS - Leibniz Institute for the Social Sciences
  • Yi Bu, Peking University
  • Guillaume Cabanac, IRIT - Université Paul Sabatier Toulouse 3
  • Caitlin Cassidy, Search Technology Inc
  • Chong Chen, Beijing Normal University
  • Guo Chen, Nanjing University of Science and Technology
  • Hongshu Chen, Beijing Institute of Technology
  • Gong Cheng, Nanjing University
  • Jian Du, Peking University
  • Edward Fox, Virgina Tech
  • Ying Guo, China University of Political Science and Law
  • Arash Hajikhani, VTT Technical Research Centre of Finland
  • Jiangen He, The University of Tennessee
  • Zhigang Hu, South China Normal University
  • Bolin Hua, Peking University
  • Ying Huang, Wuhan University
  • Yong Huang, Wuhan University
  • Yuya Kajikawa, Tokyo University of Technology
  • Vivek Kumar Singh, Banaras Hindu University, Varanasi, U.P., India
  • Chenliang Li, Wuhan Univerisity
  • Kai Li, University of Tennessee
  • Chao Lu, Hohai University
  • Shutian Ma, Tencent
  • Jin Mao, Wuhan University
  • Xianling Mao, Beijing Institute of Technology
  • Chao Min, Nanjing University
  • Wolfgang Otto, GESIS - Leibniz-Institute for the Social Sciences
  • Xuelian Pan, Nanjing University
  • Dwaipayan Roy, GESIS - Leibniz-Institute for the Social Sciences
  • Philipp Schaer, TH Köln (University of Applied Sciences)
  • Mayank Singh, Indian Institute of Technology Gandhinagar
  • Bart Thijs, ECOOM, MSI, K.U.Leuven
  • Suppawong Tuarob, Mahidol University
  • Dongbo Wang, Nanjing Agricultural University
  • Xuefeng Wan,g Beijing Institute of Technology
  • Yuzhuo Wang, Anhui University
  • Dietmar Wolfram, University of Wisconsin-Milwaukee
  • Jian Wu, Old Dominion University
  • Mengjia Wu, University of Technology Sydney
  • Tianxing Wu, Southeast University
  • Xiaolan Wu, Nanjing Normal University
  • Yanghua Xiao, Fudan University
  • Jian Xu, Sun Yat-sen university
  • Shuo Xu, Beijing University of Technology
  • Erjia Yan, Drexel University
  • Heng Zhang, Nanjing University of Science and Technology
  • Jinzhu Zhang, Nanjing University of Science and Technology
  • Xiaojuan Zhang, Southwest University
  • Yingyi Zhang, Soochow University
  • Zhixiong Zhang, National Science Library, Chinese Academy of Sciences
  • Qingqing Zhou, Nanjing Normal University
  • Yongjun Zhu, Yonsei University


  1. Chang, X., Zheng, Q. (2008). Knowledge Element Extraction for Knowledge-Based Learning Resources Organization. In: Leung, H., Li, F., Lau, R., Li, Q. (eds) Advances in Web Based Learning – ICWL 2007. ICWL 2007. Lecture Notes in Computer Science, vol 4823. Springer, Berlin, Heidelberg.
  2. Ying, D., Min, S., Jia, H., Qi, Y., Erjia, Y., Lili, L., Tamy, C. entitymetrics: measuring the impact of entities. Plos One, 2013, 8(8), e71416.
  3. Zhang, C., Mayr, P., Lu, W., & Zhang, Y. (2022). JCDL2022 workshop: extraction and evaluation of knowledge entities from scientific documents (EEKE2022). In Proceedings of the 22nd ACM/IEEE Joint Conference on Digital Libraries (JCDL '22). Association for Computing Machinery, New York, NY, USA, Article 54, 1–2.
  4. Zhang, Y., Zhang, C., Mayr, P., & Suominen, A. An editorial of “AI + informetrics”: multi-disciplinary interactions in the era of big data. Scientometrics 127, 6503–6507 (2022).


Past Proceedings & Journal Special Issues

Proceedings can be accessed at 

We have organized the related special issues on the topic of extraction and evaluation of knowledge entities in the following journals:

We have organized the related special issues on the topic of AI + Informetrics in the following journals: