ConferenceCall 2024 03 06: Difference between revisions
Ontolog Forum
(→Agenda) |
|||
| (5 intermediate revisions by the same user not shown) | |||
| Line 21: | Line 21: | ||
== Agenda == | == Agenda == | ||
* 12:00 - 12:07 '''[[GaryBergCross|Gary Berg-Cross]]''' ''Summary of some issues from Session 1 and some foundational issues for LLMs vs Cognitive/Symbolic systems'' [https:// | * 12:00 - 12:07 '''[[GaryBergCross|Gary Berg-Cross]]''' ''Summary of some issues from Session 1 and some foundational issues for LLMs vs Cognitive/Symbolic systems'' [https://bit.ly/3IMSrJ3 Slides] | ||
* 12:07 - 12:37 '''Hamed Babaei Giglou''' ''Exploring LLMs for Ontology: Ontology Learning and Ontology Matching'' | * 12:07 - 12:37 '''Hamed Babaei Giglou''' ''Exploring LLMs for Ontology: Ontology Learning and Ontology Matching'' [https://bit.ly/3IqYWAL Slides] | ||
Abstract: Large Language Models for Ontology Learning (LLMs4OL) framework utilizes Large Language Models (LLMs) for Ontology Learning (OL). LLMs have shown significant advancements in natural language processing, demonstrating their ability to capture complex language patterns in different knowledge domains. Within the LLMs4OL framework, we investigate the "Can LLMs effectively apply their language pattern capturing capability to OL, which involves automatically extracting and structuring knowledge from natural language text?" | Abstract: Large Language Models for Ontology Learning (LLMs4OL) framework utilizes Large Language Models (LLMs) for Ontology Learning (OL). LLMs have shown significant advancements in natural language processing, demonstrating their ability to capture complex language patterns in different knowledge domains. Within the LLMs4OL framework, we investigate the "Can LLMs effectively apply their language pattern capturing capability to OL, which involves automatically extracting and structuring knowledge from natural language text?" | ||
To explore LLMs in the area of OL, the first challenge is the formulation of the work and where to use LLM to be suitable for the given task. We conduct a comprehensive analysis using the zero-shot prompting method to evaluate nine different LLM model families for three main OL tasks (where our formulation to use LLMs comes in): term typing, taxonomy discovery, and extraction of non-taxonomic relations. Additionally, the evaluations encompass diverse genres of ontological knowledge, including lexicosemantic knowledge in WordNet, geographical knowledge in GeoNames, and medical knowledge in UMLS. | To explore LLMs in the area of OL, the first challenge is the formulation of the work and where to use LLM to be suitable for the given task. We conduct a comprehensive analysis using the zero-shot prompting method to evaluate nine different LLM model families for three main OL tasks (where our formulation to use LLMs comes in): term typing, taxonomy discovery, and extraction of non-taxonomic relations. Additionally, the evaluations encompass diverse genres of ontological knowledge, including lexicosemantic knowledge in WordNet, geographical knowledge in GeoNames, and medical knowledge in UMLS. | ||
The obtained empirical results show that foundational LLMs are not sufficiently suitable for ontology construction that entails a high degree of reasoning skills and domain expertise. Nevertheless, when effectively fine-tuned they just might work as suitable assistants, alleviating the knowledge acquisition bottleneck, for ontology construction. | The obtained empirical results show that foundational LLMs are not sufficiently suitable for ontology construction that entails a high degree of reasoning skills and domain expertise. Nevertheless, when effectively fine-tuned they just might work as suitable assistants, alleviating the knowledge acquisition bottleneck, for ontology construction. | ||
Within this presentation, the main focus will be given to experimented LLM variants and results analysis within the OL task. The whole project is released to the community with detailed documentation here: [https://github.com/HamedBabaei/LLMs4OL https://github.com/HamedBabaei/LLMs4OL] | Within this presentation, the main focus will be given to experimented LLM variants and results analysis within the OL task. The whole project is released to the community with detailed documentation here: [https://github.com/HamedBabaei/LLMs4OL https://github.com/HamedBabaei/LLMs4OL] | ||
| Line 36: | Line 33: | ||
Bio: Hamed Babaei Giglou is a researcher at TIB and is currently, involved in the Neural-Symbolic SCholarly InnovatioN EXTraction (SciNEXT) project in collaboration with Open Research Knowledge Graph (ORKG) project at TIB -- German National Library of Science and Technology. Hamed got a bachelor's and Master's degree in computer science and worked as an NLP Researcher for more than 3 years in the industry before joining the TIB as a PhD candidate. Currently, he is pursuing a PhD in "Computer Science: NLP and Semantic Web Technologies" under the supervision of Dr. Jennifer D'Souza and Prof. Dr. Sören Auer. His current research focuses on employing LLM in various ontology tasks, such as Ontology Learning and Ontology Matching. | Bio: Hamed Babaei Giglou is a researcher at TIB and is currently, involved in the Neural-Symbolic SCholarly InnovatioN EXTraction (SciNEXT) project in collaboration with Open Research Knowledge Graph (ORKG) project at TIB -- German National Library of Science and Technology. Hamed got a bachelor's and Master's degree in computer science and worked as an NLP Researcher for more than 3 years in the industry before joining the TIB as a PhD candidate. Currently, he is pursuing a PhD in "Computer Science: NLP and Semantic Web Technologies" under the supervision of Dr. Jennifer D'Souza and Prof. Dr. Sören Auer. His current research focuses on employing LLM in various ontology tasks, such as Ontology Learning and Ontology Matching. | ||
* 12:37 - 13:00 Discussion | * 12:37 - 13:00 Discussion | ||
* [https://bit.ly/4a4S1t8 Video Recording] | |||
* [https://youtu.be/R0Hjnqi2eT0 Youtube Video] | |||
== Conference Call Information == | == Conference Call Information == | ||
| Line 45: | Line 44: | ||
== Participants == | == Participants == | ||
* [[BobbinTeegarden|Bobbin Teegarden]] | |||
* [[FrankLoebe|Frank Loebe]] | |||
* [[GaryBergCross|Gary Berg-Cross]] | |||
* Hamed Babaei Giglou | |||
* [[KenBaclawski|Ken Baclawski]] | |||
* [[MikeBennett|Mike Bennett]] | |||
* Phil Jackson | |||
* [[RaviSharma|Ravi Sharma]] | |||
* [[ToddSchneider|Todd Schneider]] | |||
== Discussion == | == Discussion == | ||
== Foundational Ontologies == | === Foundational Ontologies === | ||
[12:21] ToddSchneider: How can existing foundational ontologies be incorporated into this work? | [12:21] ToddSchneider: How can existing foundational ontologies be incorporated into this work? | ||
== Biomedical == | === Biomedical === | ||
[12:33] DrRavi Sharma : Why NCi related results are lower, is it hat the information is highly correlated in medicine | [12:33] DrRavi Sharma : Why NCi related results are lower, is it hat the information is highly correlated in medicine | ||
== Reasoning == | === Reasoning === | ||
[12:35] ToddSchneider: Was a reasoner used to test the (logical) consistency of the resulting artifacts? | [12:35] ToddSchneider: Was a reasoner used to test the (logical) consistency of the resulting artifacts? | ||
| Line 67: | Line 75: | ||
[12:56] ToddSchneider: Logical reasoning is applied to a collection of logical and non-logical symbols, along with rules of inference (on those symbols). | [12:56] ToddSchneider: Logical reasoning is applied to a collection of logical and non-logical symbols, along with rules of inference (on those symbols). | ||
== Track D: Finance Panel == | === Track D: Finance Panel === | ||
[12:56] MikeBennett : Finance is still looking for speakers | [12:56] MikeBennett : Finance is still looking for speakers | ||
== Resources == | == Resources == | ||
* [https://bit.ly/4a4S1t8 Video Recording] | |||
* [https://youtu.be/R0Hjnqi2eT0 Youtube Video] | |||
== Previous Meetings == | == Previous Meetings == | ||
Latest revision as of 22:12, 7 March 2024
| Session | LLMs, Ontologies and KGs |
|---|---|
| Duration | 1 hour |
| Date/Time | 6 Mar 2024 17:00 GMT |
| 9:00am PST/12:00pm EST | |
| 5:00pm GMT/6:00pm CET | |
| Convener | Gary Berg-Cross |
Ontology Summit 2024 LLMs, Ontologies and KGs
Agenda
- 12:00 - 12:07 Gary Berg-Cross Summary of some issues from Session 1 and some foundational issues for LLMs vs Cognitive/Symbolic systems Slides
- 12:07 - 12:37 Hamed Babaei Giglou Exploring LLMs for Ontology: Ontology Learning and Ontology Matching Slides
Abstract: Large Language Models for Ontology Learning (LLMs4OL) framework utilizes Large Language Models (LLMs) for Ontology Learning (OL). LLMs have shown significant advancements in natural language processing, demonstrating their ability to capture complex language patterns in different knowledge domains. Within the LLMs4OL framework, we investigate the "Can LLMs effectively apply their language pattern capturing capability to OL, which involves automatically extracting and structuring knowledge from natural language text?"
To explore LLMs in the area of OL, the first challenge is the formulation of the work and where to use LLM to be suitable for the given task. We conduct a comprehensive analysis using the zero-shot prompting method to evaluate nine different LLM model families for three main OL tasks (where our formulation to use LLMs comes in): term typing, taxonomy discovery, and extraction of non-taxonomic relations. Additionally, the evaluations encompass diverse genres of ontological knowledge, including lexicosemantic knowledge in WordNet, geographical knowledge in GeoNames, and medical knowledge in UMLS.
The obtained empirical results show that foundational LLMs are not sufficiently suitable for ontology construction that entails a high degree of reasoning skills and domain expertise. Nevertheless, when effectively fine-tuned they just might work as suitable assistants, alleviating the knowledge acquisition bottleneck, for ontology construction.
Within this presentation, the main focus will be given to experimented LLM variants and results analysis within the OL task. The whole project is released to the community with detailed documentation here: https://github.com/HamedBabaei/LLMs4OL
Bio: Hamed Babaei Giglou is a researcher at TIB and is currently, involved in the Neural-Symbolic SCholarly InnovatioN EXTraction (SciNEXT) project in collaboration with Open Research Knowledge Graph (ORKG) project at TIB -- German National Library of Science and Technology. Hamed got a bachelor's and Master's degree in computer science and worked as an NLP Researcher for more than 3 years in the industry before joining the TIB as a PhD candidate. Currently, he is pursuing a PhD in "Computer Science: NLP and Semantic Web Technologies" under the supervision of Dr. Jennifer D'Souza and Prof. Dr. Sören Auer. His current research focuses on employing LLM in various ontology tasks, such as Ontology Learning and Ontology Matching.
- 12:37 - 13:00 Discussion
- Video Recording
- Youtube Video
Conference Call Information
- Date: Wednesday, 6 March 2024
- Start Time: 9:00am PST / 12:00pm EST / 6:00pm CET / 5:00pm GMT / 1700 UTC
- ref: World Clock
- Expected Call Duration: 1 hour
- Video Conference URL: https://bit.ly/48lM0Ik
- Conference ID: 876 3045 3240
- Passcode: 464312
The unabbreviated URL is: https://us02web.zoom.us/j/87630453240?pwd=YVYvZHRpelVqSkM5QlJ4aGJrbmZzQT09
Participants
- Bobbin Teegarden
- Frank Loebe
- Gary Berg-Cross
- Hamed Babaei Giglou
- Ken Baclawski
- Mike Bennett
- Phil Jackson
- Ravi Sharma
- Todd Schneider
Discussion
Foundational Ontologies
[12:21] ToddSchneider: How can existing foundational ontologies be incorporated into this work?
Biomedical
[12:33] DrRavi Sharma : Why NCi related results are lower, is it hat the information is highly correlated in medicine
Reasoning
[12:35] ToddSchneider: Was a reasoner used to test the (logical) consistency of the resulting artifacts?
[12:41] ToddSchneider: How could the results of a reasoner be incorporated in the ‘process’? That is, how could a reasoner be used as part of the ‘learning’ process?
[12:47] PhilJackson : can the system learn ontologies of verbs (actions) or adjectives (properties), as well as ontologies of nouns (classes)?
[12:51] BobbinTeegarden: Doesn't reasoning imply a logical structure, and is there a 'logical structure' assumed in a neural net process?
[12:56] ToddSchneider: Logical reasoning is applied to a collection of logical and non-logical symbols, along with rules of inference (on those symbols).
Track D: Finance Panel
[12:56] MikeBennett : Finance is still looking for speakers
Resources
Previous Meetings
Next Meetings
| Session | |
|---|---|
| ConferenceCall 2023 10 04 | Overview |
| ConferenceCall 2023 10 11 | Setting the stage |
| ConferenceCall 2023 10 18 | A look across the industry, Part 1 |
| ... further results | |