Ontolog Forum
Ontology Summit 2011: Panel Session-6 - "Integrating the Ontology Application Framework, Use Cases, Value and Metrics" - Thu 2011_03_03
Summit Theme: OntologySummit2011: Making the Case for Ontology
Session Title: Integrating the Ontology Application Framework, Use Cases, Value and Metrics
Session Chair: Dr. SteveRay (CMU)
Panelists:
- Professor MichaelGruninger (U of Toronto) - "The Ontology Application Framework"
- Mr. RexBrooks (Starbourne) - "Value Metrics"
- Mr. MikeBennett (Hypercube) - "Use Cases - how they fit into the framework"
- ==Archives==
- Abstract
- Agenda
- Prepared presentation material can be accessed by clicking on each of the links below:
- [ 1-Gruninger ] . [ 2-Brooks ] . [ 3-Bennett ] . [ 4-List-archives ]
- Audio recording of the session [ 1:23:42 ; mp3 ; 9.6 MB ]
- transcript of the online chat during the session
- Additional Resources
Abstract
OntologySummit2011 Theme: "Making the Case for Ontology"
- Session Title: Integrating the Ontology Application Framework, Use Cases, Value and Metrics
This is our 6th Ontology Summit, a joint initiative by NIST, Ontolog, NCOR, NCBO, IAOA & NCO_NITRD. The theme adopted for this Ontology Summit is: "Making the Case for Ontology."
This year's Ontology Summit seeks to address the need to provide concrete evidence of successful deployment of ontologies by examining several application domains for such examples, and in better articulating where different "strengths" of ontological representation are best applied. To support that, the summit also aims to classify the categories of applications where ontology has been, and could be, successfully applied; to identify distinct types of metrics that might be used in evaluating the return on investment in an ontology application (cost, capability, performance, etc.); to lay out some strategies for articulating a case for ontological applications; and to identify remaining challenges and roadblocks to a wider deployment of such applications that represent promising application areas and research challenges for the future. The findings of the summit will be documented in the form of a communiqué intended for public consumption.
This session begins the process of combining the excellent work done within several of the tracks. Specifically, we will hear from Track 1 - Ontology Application Framework - where Michael Grüninger will describe how they have categorized various classes of applications that use, or could use, ontology. Then, Rex Brooks (Track 3 - Value Metrics and Value Models) will show how this fits into a larger context where the applications and measures of benefits form a context that includes the different kinds of stakeholders and how different benefit (or value) metrics can be of use to different audiences and with different kinds of applications. Finally, Mike Bennett (Track 2 - Documented case studies) will show how some of the use cases we have seen fit into these frameworks.
The second half of the session will be open for discussion and questions.
See developing details on this Summit series of events at: OntologySummit2011 (home page for this summit)
Agenda
Ontology Summit 2011 - Panel Session-6
- Session Format: this is a virtual session conducted over an augmented conference call
- 1. Opening (chair) - Steve Ray [10 min.] [ presentation material ]
- 2. Panelists presentations - Michael Grüninger, Rex Brooks, Mike Bennett [15 min. each]
- 3. Q & A and open discussion [All: 60 min.] -- please refer to process above
- 4. Wrap-up / Announcements (co-chair) - Steve Ray
Proceedings
Please refer to the above
IM Chat Transcript captured during the session
see raw transcript here.
(for better clarity, the version below is a re-organized and lightly edited chat-transcript.)
Participants are welcome to make light edits to their own contributions as they see fit.
-- begin in-session chat-transcript --
[09:27] Peter P. Yim: Peter P. Yim: Welcome to the ...
Ontology Summit 2011: Panel Session-6 - Thu 2011_03_03
Summit Theme: Ontology Summit 2011: Making the Case for Ontology
Session Title: Integrating the Ontology Application Framework, Use Cases, Value and Metrics
Session Co-chairs: Dr. Steve Ray (CMU)
Panelists:
- Professor Michael Grüninger (U of Toronto) - "The Ontology Application Framework"
- Mr. Rex Brooks (Starbourne) - "Value Metrics"
- Mr. Mike Bennett (Hypercube) - "Use Cases - how they fit into the framework"
Please refer to dial-in information, agenda, and other details on the session page
at: http://ontolog.cim3.net/cgi-bin/wiki.pl?ConferenceCall_2011_03_03
anonymous morphed into Matt Hettinger
anonymous morphed into Marcy Harris
Steve Ray: Cheat sheet: *2 to mute, *3 to unmute
anonymous morphed into Randy Coleman
Michael Uschold: test
Mike Bennett: No joy
Mike Bennett: Tried the magic thing
Mike Bennett: I'll diall out and in again
anonymous morphed into Naicong Li
anonymous morphed into JulitaBermejoAlonso
Steve Ray: Sorry about that, Rex. Your voice is definitely clearer with the handset.
Nicola Guarino: Maybe we should specify "where" the functionality provided by ontologies is applied.
E.g, specification is applied to conceptualizations; classification can be applied to concepts or
indivduals...
Steve Ray: One of the "-ilities" that fell off is "capability"
Steve Ray: @MichaelGruninger: Interesting that you didn't choose "specification" in the Integration
category, as well as mapping.
Rex Brooks: @MichaelGruninger: I think of Decision Support as marshalling information more than
automated inferencing. However, if the inferencing marshall information, then I'm good with it.
Nicola Guarino: Information integration can also occur at development time, in the sense that
ontologies are used to manually integrate different conceptual models. In these cases, ontologies
can also help to recognize the impossibility of integration...
Michael Grüninger: @Nicola: "Where an ontology is used" might indeed be another dimension in the
framework
Michael Grüninger: @Nicola: Although information integration can occur at design time, I put this
into the Ontology Augmentation system category because it is being used to integrate conceptual
models before the system itself is designed. Unless you mean that conceptual models are integrated,
in which case I wonder whether this is also being used at run-time
Steve Ray: @MichaelGruninger: It would be great if you could add the answers you gave on the phone to
the wiki page, for each of the classes of applications, i.e. which dimension, who is the user, etc.
Randy Coleman: @MichaelGruninger: Have you had thoughts about multi-dimension/multi-classification
ontologies?
Michael Uschold: Let me walk people through this -
http://ontolog.cim3.net/file/work/OntologySummit2011/ApplicationFramework/OWL-Ontology/
Michael Uschold: look at:
http://ontolog.cim3.net/file/work/OntologySummit2011/ApplicationFramework/OWL-Ontology/OntologyAppli
cationFramework.pdf
Rex Brooks: @MichaelUschold: One thing I didn't think about was results v. value metrics. sometimes
the results are unfavorable and we need to capture that and learn from it., too.
Fabian Neuhaus: @ Michael Grüninger: are "problem addressed" and "benefit" different dimensions?
Wouldn't being able to address a problem be a benefit?
Michael Grüninger: @Fabian: Earlier discussions have indicated a need to distinguish the benefit an
ontology can deliver from the original problem that motivated the use of the ontology in the first
place.
Matt Hettinger: @MichaelGruninger if a set of ontologies are used at design time for a system other
than the ontology system that is being used (e.g. some line-of-business system) then the use of that
set of ontologies, that ontology system, can be (should be) considered a run-time use.
Steve Ray: @MichaelUschold: Probably the best way to decide what perspective to take with your
diagrams is to think about the original purpose of the Summit, which is to help people trying to
make the case. If a given aspect doesn't speak to that, then you won't need to include it.
Steve Ray: @MichaelUschold: We lost your audio. Can you dial back in?
Michael Uschold: how long ago did you miss me?
Steve Ray: @MichaelUschold: It was only recent. You were doing the benefits.
Fabian Neuhaus: @MichaelUschold: I think you were about to finish your presentation (when you got cut
off)
Peter P. Yim: @MichaelUschold - you should be back on now ... try your voice
Pavithra Kenjige: Generally cost benefit is analyzed during Business Case development to get
funding.. these are good points..
anonymous morphed into Bart Gajderowicz
Nicola Guarino: what is it Ajax?
Steve Ray: @Nicola: See http://en.wikipedia.org/wiki/Ajax_framework for material on Ajax.
Jim Rhyne: Ajax is a browser programming technique that ships data to the browser in XML and uses
Javascript to perform local interactions with the user, avoiding the need for roundtrips with the
http server.
Bart Gajderowicz: AJAX was developed by a UI designer, so its benefits are best demonstrated by
example. Google maps is great for that. Ontologies may not be as easily demonstrable. Actually, that
designer specifically coined the term AJAX because explaining the technology was not "selling" the
it.
Jim Rhyne: @MichaelUschold - curious about the diagramming technique shown in e6owl Legend PDF. Can
you point me to additional documentation. I work on the OWL profiles for UML.
DJimRhyne: @MichaelUschold - never mind, found it on the Semantic Arts website.
JustinCote: I think this is why we like PowerPoints
Steve Ray: @MikeBennett: It looks like many of the case study providers gave benefits, but we're
having difficulty getting all the way to actual metrics.
Mike Bennett: @SteveRay I agree, that is a challenge. I hope we can get some of the Case Study
presenters to come back and put metrics in, in line with what's now in the Metrics track.
Rex Brooks: @Steve: Sorry for jumping the gun on ya.
Steve Ray: @Rex: No problem at all.
Rex Brooks: @MikeBennett: Excellent work. Extracting these is no picnic.
Nicola Guarino: my audio got disconnected. Calling again.
Michael Grüninger: @MikeBennett: I think that the Purpose of the ontology that I presented is
synonymous with your term "Challenges"
Mike Bennett: Agreed.
Michael Uschold: I also agree, ontology features describe the ontology, as @MichaelGruninger is
saying.
Steve Ray: @MikeBennett: Do you observe good coverage over all the dimensions and aspects that
Michael Grüninger laid out, or, just as interestingly, do you see clustering of use cases around just
a few?
Peter P. Yim: @JohnSowa - can you capture your point to this chat (so it will go into the transcript)
Michael Uschold: @JohnSowa: I build the ontology mainly to help clarify my own thinking, as well as
to have more precise artifact to communicate to others. I was going round and round thinking about
many distinctions. It was easier and faster for me to draw a picture (which translated into OWL)
than write yet more text in say Word or html. I agree that many will not care about this mode of
presenting a framework.
Jim Rhyne: @JohnSowa - is there no middle ground between inspiration and CYA?
Bruce Bray: The rare disruptive technologies / "killer apps" don't need boring metrics, but the usual
incremental improvements often do in order to get funded.
Michael Grüninger: @JohnSowa: the issue is that we need a common way of specifying the use cases --
what are the relevant pieces of information that need to be included? If people have different
expectations about what use cases are, then we have not made progress.
Michael Uschold: I very much agree with Fabian's point, that we want systematic way to think about
and communicate the use cases.
Jim Rhyne: @JohnSowa - Steve Jobs is a brilliant risk taker who is supported by some of the brightest
market, usability and technology analysts around. He does not invent iPADs in a vacuum.
Michael Grüninger: We also need to battle Buzzword Bingo -- if someone says "I used an ontology for
semantic integration" (or any other buzzword), we need everyone to agree on exactly what that means.
Otherwise, people will be expecting something specific and then be disillusioned when they don't see
it.
Michael Grüninger: @JohnSowa: Of course, if we had the one case (killer app) then we would use it.
The problem is that people are using ontologies for many different reasons. Everyone has a different
killer app that they are looking for.
Matt Hettinger: @RexBrooks (and any other who may have an answer) With respect to measures / metrics.
A valid measure, at least in the relational approach to measurement, is that it is required that
there is a empirical model, a numerical model, and a mapping between the two. If these do not exist
then what is called a "measure", strictly speaking, is not a measure. Once consequence is the level
of trust in the numbers produced. For the measures discussed, are there empirical models, numerical
models and mappings. (I'm not familiar with other approaches to measurement)
Rex Brooks: @MattHettinger: I agree, and yes, for measurements such as length of time for
implementation, response time in the logs for performance, uptime v. downtime, etc, there is a
correct and trustable relationship. However for qualitative measurements like Customer Satisfaction,
which I expect many of us would not believe if it was just thrown at us to take at face value, it is
more difficult. I'm not saying such qualitative measurements should be accepted. Quite the contrary.
I think they need to be dissected and analyzed before any trust is achieved.
Rex Brooks: Oh yeah, they have to be valid after the analysis, or else require a boatload of
corroborating evidence.
Steve Ray: Risk is another metric we might have overlooked - certainly important to many decision
makers.
Mike Bennett: @Michael there is a real risk of this in some of the ISO WGs (which shall remain
nameless), where some technical folks are starting to describe semantics / ontology / OWL as some
kind of magic paint.
Mike Bennett: @SteveRay Definitely. At the business level, every business case comes down to one or
other of cost and risk. What else is there?
Michael Uschold: This conversation is more about strategy than it is about integrating OAF, Value
Metrics and Cases.
Rex Brooks: @MichaelUschold: true, maybe you could put your hand and address "measuresBenefit?"
Mike Bennett: Interoperability (business problem): Conceptual model (solution)?
Matt Hettinger: The killer app, from my perspective, is (inter-)enterprise architecture, as in both
my research and EA practice, all of the use cases, to some degree, can be entailed by EA
Pavithra Kenjige: Many people did not like the abbreviation "OAF" .. can we do something about that?
John F. Sowa: I agree that interoperability is essential.
Bart Gajderowicz: I 2nd the need for explaining the benefits of "semantics". Often people reject
semantics based on individual implementations like RDF. Before people realize the benefits of
ontologies, I often hear that the same problem can be solved using existing and less complex
technologies. I hear that even Siri "can be done" using data mining techniques.
Ken Baclawski: @BartGajderowicz: Are ontologies the "more complex" technology (compared with data
mining, for example)? If that is true, perhaps we need to address the way ontologies are perceived.
Data mining is hardly a simple technology, and ad hoc solutions to interoperability are very
complex.
Mike Bennett: @KenBaclawski one issue on perception: could also say that all apps have an ontology
anyway, we are about managing ontologies, having an explicit ontology in some formal notation etc.
i.e. simplifying the (existing) problem of ontologies
Michael Uschold: @KenBaclawski agree that ontology technology can reduce complexity in systems. Even
if ontology technology is complex compared to other approaches (when viewed on their own), sometimes
it takes a complex technology to simplify a greater whole.
Ken Baclawski: @MichaelUschold: That is a good approach to dealing with the perception problem, but
we need to try addressing it by a number of routes.
Mike Bennett: Also conceptual model as an industry standard, which perhaps we haven't explicitly
mentioned
Pavithra Kenjige: Interoperability -? We need businesses that used Ontology to do the case study
regarding interoperability//
John F. Sowa: For my answer, see slides 3 to 7 of http://www.jfsowa.com/talks/iss.pdf
Nicola Guarino: A general question I have is: how the ontology quality affects the benefits achieved
for each of these use cases?
Rex Brooks: @Nicola: Yes, exactly. Ideas?
Nicola Guarino: @RexBrooks: for example, in some cases a lightweight ontology is enough, in other
cases it doesn't work at all...
Michael Uschold: @SteveRay: IHMO the real reason there are few case studies with metrics is because
it is hard to have enough of the variables controlled to get any meaningful measures. Software
projects do not lend themselves to this.
Leo Obrst: In addition to benefits, you need to have costs, but also relative costs, which can be
measured with respect to other metrics. E.g., degree of precision. Technology A vs. Technology B vs.
ontologies may all eventually provide the same level of precision (of results, of transactions,
etc.), but require much different costs to do so.
Rex Brooks: @LeoObrst: Good points.
Mike Bennett: @MichaelUschold - that is a very good point. Some industries are not native to systems
development and won't have the metrics of what they did before, that did not work so well, by
definition.
Nicola Guarino: I was just discussing with Rex about the need to evaluate the various case studies
with respect to the quality and depth of the ontology they used
Leo Obrst: We've found that people who have not gone through the XML "revolution" do not understand
the value of semantics. When they adopt XML and find out it doesn't provide what they need, they
begin to understand the difference between structural/syntactic approaches and semantic approaches.
Mike Bennett: @LeoObrst - that exactly reflects the financial services industry experience
Bobbin Teegarden: one of the major breakthroughs using ontologies/semantic web is the ability to
visualize whole systems and interactions, getting a 'holonic' view of systems of systems -- looking
for this at the granularity of 'application' may be missing the point
Matt Hettinger: @BobbinTeegarden - yes!
Peter P. Yim: - session ended: 11:10 --
-- end of in-session chat-transcript --
- Further Question & Remarks - please post them to the [ ontology-summit ] listserv
- all subscribers to the previous summit discussion, and all who responded to today's call will automatically be subscribed to the [ ontology-summit ] listserv
- if you are already subscribed, post to <ontology-summit [at] ontolog.cim3.net>
- (if you are not yet subscribed) you may subscribe yourself to the [ ontology-summit ] listserv, by sending a blank email to <ontology-summit-join [at] ontolog.cim3.net> from your subscribing email address, and then follow the instructions you receive back from the mailing list system.
- please email <peter.yim@cim3.com> if you have any question.
Audio Recording of this Session
- To download the recording of the session, click here
- the playback of the audio files require the proper setup, and an MP3 compatible player on your computer.
- Conference Date and Time: 3-Feb-Mar 9:38am~11:10am PST
- Duration of Recording: 1 Hour 24 Minutes
- Recording File Size: 9.6 MB (in mp3 format)
- suggestions:
- its best that you listen to the session while having the respective presentations opened in front of you. You'll be prompted to advance slides by the speaker.
- Take a look, also, at the rich body of knowledge that this community has built together, over the years, by going through the archives of noteworthy past Ontolog events. (References on how to subscribe to our podcast can also be found there.)
Additional Resources
- Homepage of OntologySummit2011
- Ontology Summit 2011 Launch Event - ConferenceCall_2011_01_20
- "Ontology Application Framework" session - ConferenceCall_2011_02_03
- "Ontology Applications & Use Cases - I" - ConferenceCall_2011_01_27
- "Ontology Applications & Use Cases - II" - ConferenceCall_2011_02_10
- "Value Metrics, Value Models and the Value Proposition" - ConferenceCall_2011_02_17
- "Strategies for 'Making the Case'" - ConferenceCall_2011_02_24
- [ontology-summit] mailing list archives - http://ontolog.cim3.net/forum/ontology-summit/
- to subscribe to this discussion list: send a blank message from your subscribing email address to <ontology-summit-join[at]ontolog.cim3.net> or visit http://ontolog.cim3.net/mailman/listinfo/ontology-summit/ and subscribe yourself there
- Homepage of this annual Summit series - see: OntologySummit
For the record ...
How To Join (while the session is in progress)
- 1. Dial in with a phone: http://ontolog.cim3.net/cgi-bin/wiki.pl?ConferenceCall_2011_03_03#nid2OHV
- 2. Open chat in a new browser window: http://webconf.soaphub.org/conf/room/ontolog_20110303
- 3. Point your browser to the presentation material for each speaker here: http://ontolog.cim3.net/cgi-bin/wiki.pl?ConferenceCall_2011_03_03#nid2OHK
- or, 3.1 (access our shared-screen vnc server, if you are not behind a corporate firewall)
Conference Call Details
- Date: Thursday 3-Mar-2011
- Start Time: 9:30am PST / 12:30pm EST / 6:30pm CET / 5:30pm GMT / 17:30 UTC
- see world clock for other time zones
- do note that this is 1 hour earlier than the usual 1:30 EST Ontolog event time!
- Expected Call Duration: ~2.0 hours
- Dial-in Number:
- from a US telephone (US): +1-218-844-8060 (domestic long distance cost will apply)
- When calling in from a phone, use Conference ID: "4389979#"
- from Europe, call:
- Austria 0820-4000-1577
- Belgium 070-35-9992
- France 0826-100-280
- Germany 01805-00-7642
- Ireland 0818-270-037
- Italy 848-390-179
- Spain 0902-886-056
- Switzerland 0848-560-327
- UK 0844-581-9148
- callers from other countries please dial into either one of the US or European numbers
- Shared-screen support (VNC session), if applicable, will be started 5 minutes before the call at: http://vnc2.cim3.net:5800/
- view-only password: "ontolog"
- if you plan to be logging into this shared-screen option (which the speaker may be navigating), and you are not familiar with the process, please try to call in 5 minutes before the start of the session so that we can work out the connection logistics. Help on this will generally not be available once the presentation starts.
- people behind corporate firewalls may have difficulty accessing this. If that is the case, please download the slides above and running them locally. The speaker(s) will prompt you to advance the slides during the talk.
- Discussions and Q & A:
- (Unless the conference host has already muted everyone) Please mute your phone, by pressing "*2" on your phone keypad, when a presentation is in progress. To un-mute, press "*3"
- You can type in your questions or comments through the browser based chat session by:
- pointing a separate browser tab (or window) to http://webconf.soaphub.org/conf/room and enter: Room="ontolog_20110303" and My Name="Your Own Name" (e.g. "JaneDoe")
- or point your browser to: http://webconf.soaphub.org/conf/room/ontolog_20110303
- instructions: once you got access to the page, click on the "settings" button, and identify yourself (by modifying the Name field). You can indicate that you want to ask a question verbally by clicking on the "hand" button, and wait for the moderator to call on you; or, type and send your question into the chat window at the bottom of the screen.
- (when everyone is muted) If you want to speak or have questions or remarks to make, please "raise your hand (virtually)" by click on the "hand button" (lower right) on the chat session page. You may speak when acknowledged by the speaker or the session moderator (again, press "*3" on your phone to unmute). Test your voice and introduce yourself first before proceeding with your remarks, please. (Please remember to click on the "hand button" again (to lower your hand) and press "*2" on your phone to mute yourself after you are done speaking.)
- thanks to the soaphub.org folks, one can now use a jabber/xmpp client (e.g. gtalk) to join this chatroom. Just add the room as a buddy - (in our case here) ontolog_20110303@soaphub.org ... Handy for mobile devices!
- Please review our Virtual Session Tips and Ground Rules - see: VirtualSpeakerSessionTips
- RSVP to peter.yim@cim3.com appreciated.
- This session, like all other Ontolog events, is open to the public. Information relating to this session is shared on this wiki page: http://ontolog.cim3.net/cgi-bin/wiki.pl?ConferenceCall_2011_03_03
- Please note that this session will be recorded, and the audio archive is expected to be made available as open content to our community membership and the public at-large under our prevailing open IPR policy.
Attendees
- Attended:
- Steve Ray
- Michael Grüninger
- Rex Brooks
- Mike Bennett
- Nicola Guarino
- Peter P. Yim
- Al Stevens
- Alden Dima
- Randy Coleman
- Yuriy Milov
- Matt Hettinger
- Jim Rhyne
- Naicong Li
- Kathleen Ellis
- Pavithra Kenjige
- Amanda Vizedom
- Bobbin Teegarden
- Bruce Bray
- Dustin Cote
- Fabian Neuhaus
- JulitaBermejoAlonso
- Marcy Harris
- Michael Uschold
- Rick Kiefer
- John F. Sowa
- Bob Smith
- Leo Obrst
- Ken Baclawski
- Bart Gajderowicz
- Expecting:
-
- ... if you are coming to the session, please add your name above (plus your affiliation, if you aren't already a member of the community) above; or e-mail <peter.yim@cim3.com> so that we can reserve enough resources to support everyone's participation. ...
- Regrets:
- Yu Lin
- Matthew West
- ...