Abstract: This keynote concerns the emergence of large language models, most recently and popularly embodied ChatGPT. These models have been trained using deep learning algorithms on a massive corpus of text data collected from sources across the web. These models demand a fundamental change in the conduct of scientometric research. In this speech I describe why I think such changes are necessary, and how such research changes may be enacted.

Dr. Scott W. Cunningham is a Professor of Technology Policy at the University of Strathclyde in Glasgow, UK. He is one of the editor-in-chief of the scientific journal Technological Forecasting and Social Change. He is interested in data and governance, and has published both on data use in the public sector, as well as the governance of new technologies including machine learning. Prior to joining Strathclyde he was appointed at the Delft University of Technology. Prior to his academic career he worked as a data scientist in industry, earning national and international patents for his work on automation and pricing. He has previously published books on technology mining, technology management, and technology forecasting with Alan Porter (professor emeritus, Georgia Tech). His current book, commissioned by Edward Elgar (2025) concerns the politics of technological change.

2nd Keynote: Large Language Models for Information Retrieval and Extraction

Abstract: Large language models (LLMs) have changed AI and in many ways determine what AI is and can do. Though there have been many applications and uses of LLMs, we will discuss what LLMs mean for information retrieval and extraction. Already an LLM is used in Google ranking. How else can LLMs be used in these areas and what can and can’t they do.

Dr. C. Lee Giles is the David Reese Professor in Information Sciences and Technology, graduate college Professor of Computer Science and Engineering, courtesy Professor of Supply Chain and Information Systems, and Director of the Intelligent Systems Research laboratory at the Pennsylvania State University, University Park, PA. His research areas include intelligent information processing systems, text processing, deep learning, digital libraries, and novel research search engines. He was a cocreater and now director of the scholarly digital library search engine, CiteSeer. He is a Fellow of the ACM, IEEE, and International Neural Network Society (INNS). He is the recipient of the INNS Gabor prize, IEEE CIS Neural Networks Pioneer Award, NFAIS Miles Conrad Award, and twice the IBM Distinguished Faculty Award. He has published over 600 peer reviewed conference papers and journal articles with collaborators and students, including publications in Science, Nature, PLoS ONE, Proceedings of the National Academy of Sciences, and many other conference and journal venues. His publications have over 57,000 citations and an h-index of 115, according to Google Scholar.. He can be reached at: https://clgiles.ist.psu.edu/index.shtml.

Sessions

The workshop will be held on June 26~27, 2023 (Beijing Time), and specific activities include keynotes, paper presentations and a poster session.

June, 6:30am - 12: 20am (June 26, 8:30 pm ~ June 27, 2:20 am, Beijing Time)
UTC/GMT-6	Beijing Time		Speaker	Session Chair
6:00-6:30, (June 26)	20:00-20:30, (June 26)	Connection setup: we will provide details
6:30-6:40	20:30-20:40	Openning Remarks  (Online)	Co-Chairs of EEKE-AII2023 (Chengzhi Zhang, Yi Zhang, Philipp Mayr, Wei Lu, Arho Suominen, Haihua Chen, and Ying Ding)
6:40-7:10	20:40-21:10	Session 1: Poster (Online)		Chair: Hongshu Chen
6:40-6:50	20:40-20:50	An Approach for Identifying Complementary Patents Based on Deep Learning	Jinzhu Zhang and Jialu Shi
6:50-7:00	20:50-21:00	Functional Structure Recognition of Scientific Documents in Information Science	Dayu Yan, Si Shen and Dongbo Wang
7:00-7:10	21:00-21:10	Linkages among Science, Technology, and Industry	Shuo Xu, Zhen Liu and Xin An
7:10-8:00	21:10-22:00	Session 2: Entity Extraction and Applications  (Online)		Chair: Yingyi Zhang
7:10-7:30	21:10-21:30	The Relationship of Interdisciplinarity, Entity Features and Clinical Translation Potential of COVID-19 Papers	Shuang Chen and Chunli Liu
7:30-7:45	21:30-21:45	LLM-based Entity Extraction Is Not for Cybersecurity	Maxime Würsch, Andrei Kucharavy, Dimitri Percia David and Alain Mermoud
7:45-8:00	21:45-22:00	Characterizing Emerging Technologies of Global Digital Humanities Using Scientific Method Entities	Shaojian Li and Chengxi Yan
Break
8:05-9:05	21:45-22:00	Keynote 1: Scientometrics in the Era of Large Language Models  (Online presentation + Onsite)	Scott W. Cunningham	Chair: Yi Zhang
Break
9:10-10:15	23:10(June 26)-00:15(June 27)	Session 3: AI + Informetrics (Online presentation + Onsite)		Chair: Mengjia Wu
9:10-9:30	23:10-23:30	Identifying Potential Sleeping Beauties Based on Dynamic Time Warping Algorithm and Citation Curve Benchmarking	Zewen Hu, Yu Chen and Jingjing Cui.
9:30-9:45	23:30-23:45	Scientific knowledge combination in networks: new perspectives on analyzing knowledge absorption and integration	Hongshu Chen, Jingkang Liu and Zikai Liu.
9:45-10:00	23:45-24:00	UnScientify: Detecting Scientific Uncertainty in Scholarly Full Text	Panggih Kusuma Ningrum, Philipp Mayr and Iana Atanassova.
10:00-10:15	0:00-0:15(June 27)	Sentiment Classification of Scientific Citation Based on Modified BERT Attention by Sentiment Dictionary	Dahai Yu and Bolin Hua
Break
10:20-11:20	00:20-01:20 (June 27)	Keynote 2: Large Language Models and Information Extraction   (Online presentation + Onsite)	C. Lee Giles	Chair:  Jian Wu
Break
11:25-12:15(June 27)	01:25-02:15(June 27)	Session 4: EEKE + AII Onsite Session (Onsite presentation + Online)		Chair:  Haihua Chen
11:25-11:45	01:25-01:45	ClaimDistiller: Scientific Claim Extraction with Supervised Contrastive Learning	Xin Wei, Md Reshad Ul Hoque, Jian Wu, Jiang Li
11:45-12:05	01:45-02:05	Forecasting Future Topic Trends in the Blockchain Domain: Using Graph Convolutional Network	Yejin Park, Seonkyu Lim, Changdai Gu and Min Song
12:05-12:15	02:05-02:15	How does AI assist scientific research domains? Evidence based on 26 millions research articles	Qianqian Xu, Jie Meng, Jiangen He and Wen Lou.
12:15-12:20	02:15-02:20	Greeting Notes of EEKE2022	Co-Chairs of EEKE-AII2023 (Chengzhi Zhang, Yi Zhang, Philipp Mayr, Wei Lu, Arho Suominen, Haihua Chen, and Ying Ding)
12:20	02:20	End of workshop

Submission Information

Regular papers: All submissions must be written in English, following the ACM Proceedings template (10 pages for full papers and 4 pages for short papers exclusive of unlimited pages for references) and should be submitted as PDF files to EasyChair.

Poster & demonstration: We welcome submissions detailing original, early findings, works in progress and industrial applications of knowledge entities extraction ande evaluation for a special poster session, possibly with a 2-minute presentation in the main session. Some research track papers will also be invited to the poster track instead, although there will be no difference in the final proceedings between poster and research track submissions. These papers should follow the same format as the research track papers but can be shorter (2 pages for poster and demo papers).

Submit a paper

All submissions will be reviewed by at least two independent reviewers. Please be aware of the fact that at least one author per paper needs to register for the workshop and attend the workshop to present the work. In case of no-show the paper (even if accepted) will be deleted from the proceedings and from the program.

Workshop proceedings will be deposited online in the CEUR workshop proceedings publication service. This way the proceedings will be permanently available and citable (digital persistent identifiers and long term preservation).

Special Issue

Accepted submissions will be invited to submit to our special issue in Journal of Informetrics. More detailed information about this special issue can be visited at: https://eeke-workshop.github.io/2023/si-eeke-aii.html.

Important Dates

All dates are Anywhere on Earth (AoE).

Deadline for submission: May, 10, 2023
Notification of acceptance: June, 1, 2023
Camera ready: June, 20, 2023
Workshop: June, 26, 2023

Main Organising Committee

Chengzhi Zhang (zhangcz@njust.edu.cn) is a professor of Department of Information Management, Nanjing University of Science and Technology, China. He received his PhD degree of Information Science from Nanjing University, China. He has published more than 100 publications, including JASIST, Aslib JIM, JOI, OIR, SCIM, ACL, NAACL, etc. His current research interests include scientific text mining, knowledge entity extraction and evaluation, social media mining. He serves as Editorial Board Member and Managing Guest Editor for 10 international journals (Patterns, IPM, OIR, Aslib JIM, TEL, JDIS, DIM, DI, etc.) and PC members of several international conferences in fields of natural language process and scientometrics. (https://chengzhizhang.github.io/)

Yi Zhang (yi.zhang@uts.edu.au) works as a Senior Lecturer at the Australian Artificial Intelligence Institute, University of Technology Sydney. He holds dual Ph.D. degrees in Management Science & Engineering and in Software Engineering. His research interests align with intelligent bibliometrics - incorporating artificial intelligence and data science techniques with bibliometric indicators for broad science, technology & innovation studies. He is the recipient of the 2019 Discovery Early Career Researcher Award granted by the Australian Research Council. He serves as the Associate Editor for Technol. Forecast. & Soc. Change, the Editorial Board Member for the IEEE Trans. Eng. Manage., and the Advisory Board Member for the International Center for the Study of Research. (https://www.uts.edu.au/staff/yi.zhang)

Philipp Mayr ( philipp.mayr@gesis.org) is a team leader at the GESIS - Leibniz-Institute for the Social Sciences department Knowledge Technologies for the Social Sciences (WTS). He received his PhD in applied informetrics and information retrieval from the Berlin School of Library and Information Science at Humboldt University Berlin. He has published in top conferences and prestigious journals in the areas informetrics, information retrieval and digital libraries. His research group focuses on methods and techniques for interactive information retrieval and data set search. He was the main organizer of the BIR workshops at ECIR 2014-2021 and the BIRNDL workshops at JCDL 2016 and SIGIR 2017-2019. (https://philippmayr.github.io/)

Wei Lu (weilu@whu.edu.cn) is a professor of School of Information Management and director of Information Retrieval and Knowledge Mining Center, Wuhan University. He received his PhD degree of Information Science from Wuhan University, China. His current research interests include information retrieval, text mining, QA etc. He has papers published on SIGIR, Information Sciences, JASIT, Journal of Information Science etc. He serves as diverse roles (e.g., Associate Editor, Editorial Board Member, and Managing Guest Editor) for several journals. (http://39.103.203.133/member/4)

Arho Suominen (Arho.Suominen@vtt.fi) is Principal Scientist at the VTT Technical Research Centre of Finland and Industrial professor at Tampere University (Finland). Dr. Suominen’s research focuses on qualitative and quantitative assessment of innovation systems with a special focus on quantitative methods. His prior research has been funded by the European Commission via H2020, Academy of Finland, Finnish Funding Agency for Technology, Turku University Foundation and the Fulbright Center Finland. Through the Fulbright program, he worked as Visiting Scholar at the School of Public Policy at the Georgia Institute of Technology. Dr. Suominen has a Doctor of Science (Tech.) degree from the University of Turku and holds an Officers basic degree from the National Defence University of Finland. (https://cris.vtt.fi/en/persons/arho-suominen)

Haihua Chen (haihua.chen@unt.edu)is a clinical assistant professor in the Department
of Information Science at the University of North Texas. He has expertise in applied data science, natural language processing, information retrieval, and text mining. He co-authored more than 40 publications in academic venues in both information science and computer science. He is serving as co-editor for The Electronic Library, the guest editor of Information Discovery & Delivery and Frontiers in Big Data special issues, and the reviewer for 14 peer reviewed journals and several international conferences. (https://iia.ci.unt.edu/haihua-chen/)

Ying Ding (ying.ding@austin.utexas.edu)is Bill & Lewis Suit Professor at School of Information, University of Texas at Austin. She has been involved in various NIH, NSF and European-Union funded projects. She has published 240+ papers in journals, conferences, and workshops, and served as the program committee member for 200+ international conferences. She is the co-editor of book series called Semantic Web Synthesis by Morgan & Claypool publisher, the co-editor-in-chief for Data Intelligence published by MIT Press and Chinese Academy of Sciences, and serves as the editorial board member for several top journals in Information Science and Semantic Web. Her current research interests include data-driven science of science, AI in healthcare, Semantic Web, knowledge graph, data science, scholarly communication, and the application of Web technologies. (https://yingding.ischool.utexas.edu/)

Programme Committee

• Alireza Abbasi, University of New South Wales (Canberra)
  
• Andrea Scharnhorst, DANS-KNAW
  
• Iana Atanassova, CRIT, Université de Bourgogne Franche-Comté
  
• Marc Bertin, Université Claude Bernard Lyon 1
  
• Katarina Boland, GESIS - Leibniz Institute for the Social Sciences
  
• Yi Bu, Peking University
  
• Guillaume Cabanac, IRIT - Université Paul Sabatier Toulouse 3
  
• Caitlin Cassidy, Search Technology Inc
  
• Chong Chen, Beijing Normal University
  
• Hongshu Chen, Beijing Institute of Technology
  
• Gong Cheng, Nanjing University
  
• Jian Du, Peking University
  
• Edward Fox, Virgina Tech
  
• Ying Guo, China University of Political Science and Law
  
• Arash Hajikhani, VTT Technical Research Centre of Finland
  
• Saeed-Ul Hassan, Information Technology University
  
• Jiangen He, The University of Tennessee
  
• Zhigang Hu, Dalian University of Technology
  
• Bolin Hua, Peking University
  
• Ying Huang, Wuhan University
  
• Yuya Kajikawa, Tokyo University of Technology
  
• Vivek Kumar Singh, Banaras Hindu University, Varanasi, U.P., India
  
• Chenliang Li, Wuhan Univerisity
  
• Kai Li, Renmin University of China
  
• Chao Lu, Hohai University
  
• Shutian Ma, Tencent
  
• Jin Mao, Wuhan University
  
• Xianling Mao, Beijing Institute of Technology
  
• Chao Min, Nanjing University
  
• Wolfgang Otto, GESIS - Leibniz-Institute for the Social Sciences
  
• Xuelian Pan, Nanjing University
  
• Dwaipayan Roy, GESIS - Leibniz-Institute for the Social Sciences
  
• Philipp Schaer, TH Köln (University of Applied Sciences)
  
• Mayank Singh, Indian Institute of Technology Gandhinagar
  
• Bart Thijs, ECOOM, MSI, K.U.Leuven
  
• Suppawong Tuarob, Mahidol University
  
• Dongbo Wang, Nanjing Agricultural University
  
• Xuefeng Wan,g Beijing Institute of Technology
  
• Yuzhuo Wang, Nanjing University of Science and Technology
  
• Dietmar Wolfram, University of Wisconsin-Milwaukee
  
• Jian Wu, Old Dominion University
  
• Mengjia Wu, University of Technology Sydney
  
• Tianxing Wu, Southeast University
  
• Xiaolan Wu, Nanjing Normal University
  
• Yanghua Xiao, Fudan University
  
• Jian Xu, Sun Yat-sen university
  
• Shuo Xu, Beijing University of Technology
  
• Erjia Yan, Drexel University
  
• Heng Zhang, Nanjing University of Science and Technology
  
• Jinzhu Zhang, Nanjing University of Science and Technology
  
• Xiaojuan Zhang, Southwest University
  
• Yingyi Zhang, Soochow University
  
• Zhixiong Zhang, National Science Library, Chinese Academy of Sciences
  
• Qingqing Zhou, Nanjing Normal University
  
• Yongjun Zhu, Yonsei University
  

References

1. Chang, X., Zheng, Q. (2008). Knowledge Element Extraction for Knowledge-Based Learning Resources Organization. In: Leung, H., Li, F., Lau, R., Li, Q. (eds) Advances in Web Based Learning – ICWL 2007. ICWL 2007. Lecture Notes in Computer Science, vol 4823. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-78139-4_10
2. Ying, D., Min, S., Jia, H., Qi, Y., Erjia, Y., Lili, L., Tamy, C. entitymetrics: measuring the impact of entities. Plos One, 2013, 8(8), e71416. https://doi.org/10.1371/journal.pone.0071416
3. Zhang, C., Mayr, P., Lu, W., & Zhang, Y. (2022). JCDL2022 workshop: extraction and evaluation of knowledge entities from scientific documents (EEKE2022). In Proceedings of the 22nd ACM/IEEE Joint Conference on Digital Libraries (JCDL '22). Association for Computing Machinery, New York, NY, USA, Article 54, 1–2. https://doi.org/10.1145/3529372.3530917
4. Zhang, Y., Zhang, C., Mayr, P., & Suominen, A. An editorial of “AI + informetrics”: multi-disciplinary interactions in the era of big data. Scientometrics 127, 6503–6507 (2022). https://doi.org/10.1007/s11192-022-04561-w

Links

• Workshop on Extraction and Evaluation of Knowledge Entities from Scientific Documents (EEKE)
• Workshop on AI + Informetrics (AII)
  

Past Proceedings & Journal Special Issues

Proceedings can be accessed at http://ceur-ws.org/. We have organized three special issues on the topic of extraction and evaluation of knowledge entities in the Journal of Data and Information Science, Data and Information Management, Aslib Journal of Information Management and Scientometrics respectively. The first AII workshop was held at iConference2021 and the second was held at IP&MC 2022 Annual Conference. We have organized two special issues on the topic of AI + Informetrics in the Scientometrcis and Information Processing and Managerment respectively.