Actions

Ontolog Forum

Ontology Summit 2013: Panel Session-07 - Thu 2013-02-28

Summit Theme: "Ontology Evaluation Across the Ontology Lifecycle"

Summit Track Title: Track-B: Extrinsic Aspects of Ontology Evaluation

Session Topic: "Extrinsic Aspects of Ontology Evaluation-II"

  • Session Co-chairs: Dr. ToddSchneider (Raytheon) and Mr. TerryLongstreth (Ind. Consultant) -

Panelists / Briefings:

  • Dr. ToddSchneider (Raytheon) & Mr. TerryLongstreth (Ind. Consultant) - "Trinsics across the Engineering Lifecycle" slides
  • Dr. JoaoPauloAlmeida (Federal University of Espírito Santo, Brazil) - "Assessing Ontologies via Simulation" slides
  • Mr. KeithSilliman (Ind. Consultant) - "System Testing in a Data World" slides
  • Dr. AmandaVizedom (Ind. Consultant) - "From Business Requirements to Ontology Evaluation: A Process Outline" slides

Archives

Abstract

OntologySummit2013 Session-07: "Extrinsic Aspects of Ontology Evaluation-II"

This is our 8th Ontology Summit, a joint initiative by NIST, Ontolog, NCOR, NCBO, IAOA & NCO_NITRD with the support of our co-sponsors. The theme adopted for this Ontology Summit is: "Ontology Evaluation Across the Ontology Lifecycle."

Currently, there is no agreed methodology for development of ontologies, and there are no universally agreed metrics for ontology evaluation. At the same time, everybody agrees that there are a lot of badly engineered ontologies out there, thus people use -- at least implicitly -- some criteria for the evaluation of ontologies.

During this Ontology Summit, we seek to identify best practices for ontology development and evaluation. We will consider the entire lifecycle of an ontology -- from requirements gathering and analysis, through to design and implementation. In this endeavor, the Summit will seek collaboration with the software engineering and knowledge acquisition communities. Research in these fields has led to several mature models for the software lifecycle and the design of knowledge-based systems, and we expect that fruitful interaction among all participants will lead to a consensus for a methodology within ontological engineering. Following earlier Ontology Summit practice, the synthesized results of this season's discourse will be published as a Communique.

At the Launch Event on 17 Jan 2013, the organizing team provided an overview of the program, and how we will be framing the discourse around the theme of of this OntologySummit. Today's session is one of the events planned.

In this 7th virtual panel session of the Summit, we will invite several experts, who have been giving a lot of thoughts on this subject matter over the years, to join us on the panel and share their insights.

More details about this Ontology Summit is available at: OntologySummit2013 (homepage for this summit)

Briefings

  • Dr. ToddSchneider (Raytheon) & Mr. TerryLongstreth (Ind. Consultant) - "Trinsics across the Engineering Lifecycle" slides
Abstract: We address a summit omission to date with a challenge.
  • Dr. JoaoPauloAlmeida (Federal University of Espírito Santo, Brazil) - "Assessing Ontologies via Simulation" slides
Abstract: This talk will present our approach to facilitate the validation of

OntoUML models by transforming these into formal specifications in the logic-based language Alloy and using its analyzer to generate and visualize instances of the model. By allowing the observation of sequences of snapshots of model instances, the dynamics of entity creation, classification, association and destruction are revealed. This confronts the modelers with the implications of modeling choices and allows them to uncover mistakes or gain confidence in the quality of conceptual models. We are specifically interested in assessing the correspondence of what is stated in an OntoUML model and what was the original intention of the modeler or domain expert.

  • Mr. KeithSilliman (Ind. Consultant) - "System Testing in a Data World" slides
Abstract: ... This presentation will provide some insight into the role of performance engineering in the system test process. Performance testing of traditional computer systems will be reviewed. Thoughts about performance testing of "black boxes" are provided as well as some concluding observations.
  • Dr. AmandaVizedom (Ind. Consultant) - "From Business Requirements to Ontology Evaluation: A Process Outline" slides
Abstract:
This presentation applies some general software engineering discipline to ontology projects and to technology projects that incorporate ontologies. The result is primarily a process description, with an example process walk-through. The process description focuses on the ontology-specific aspects. However, since the intended audience (ontology & semantic technology developers, project planners and managers) may have minimal familiarity with some of the general concepts borrowed from software engineering and technical project management, some additional explanation is provided where useful for this audiences' understanding.
Ontology projects, and semantic technology projects in general, do not typically happen for their own sakes. They happen because someone has an operational problem, a business problem, and someone thinks that some semantic technology, incorporating but typically not limited to one or more ontologies, can contribute enough to a solution to make them worthwhile. Even ontological research, not tied to specific instances of business problems, is normally motivated by (and its funding justified with reference to) some operational solutions that could be enabled or improved by the research results. There is, however, currently a significant knowledge gap concerning the relationship between business needs and the technical characteristics of ontologies and ontology-based solutions. There is also a gap in the perception of ontologies as technical artifacts, of ontology design and development as technical processes. As a result, good engineering practice are often not applied to ontology projects, or semantic technology projects in which ontologies are incorporated.
In practice, semantic technology projects too often lack sufficient technical specification and direction to have a good chance at providing the desired solutions. Many projects that include development and/or incorporation of ontologies go forward without developing, or even thinking about, technical requirements for those ontologies. As a consequence, development of ontologies for these projects, or selection of existing ontologies for use in these projects, often proceeds without any meaningful guidelines. Ontology developers and selectors, depending on their level of experience, may have some notions of what are minimal quality (suitability) requirements in general, but they lack a set of well-grounded requirements for ontologies suitable for their own project specifically. Correspondingly, Ontology Evaluation is often omitted entirely; when performed, its value is limited by the lack of ontology requirements with respect to which ontologies can be evaluated. This presentation outlines a process for filling these gaps, providing technical requirements and direction to a project, and keeping ontology work on track by evaluating ontologies against the requirements so identified.

Agenda

OntologySummit2013 - Panel Session-07

  • Session Format: this is a virtual session conducted over an augmented conference call

Proceedings

Please refer to the above

IM Chat Transcript captured during the session

see raw transcript here.

(for better clarity, the version below is a re-organized and lightly edited chat-transcript.)

Participants are welcome to make light edits to their own contributions as they see fit.

-- begin in-session chat-transcript --

-- end of in-session chat-transcript --

  • Further Question & Remarks - please post them to the [ ontology-summit ] listserv
    • all subscribers to the previous summit discussion, and all who responded to today's call will automatically be subscribed to the [ ontology-summit ] listserv
    • if you are already subscribed, post to <ontology-summit [at] ontolog.cim3.net>
    • (if you are not yet subscribed) you may subscribe yourself to the [ ontology-summit ] listserv, by sending a blank email to <ontology-summit-join [at] ontolog.cim3.net> from your subscribing email address, and then follow the instructions you receive back from the mailing list system.
    • (in case you aren't already a member) you may also want to join the ONTOLOG community and be subscribed to the [ ontolog-forum ] listserv, when general ontology-related topics (not specific to this year's Summit theme) are discussed. Please refer to Ontolog membership details at: http://ontolog.cim3.net/cgi-bin/wiki.pl?WikiHomePage#nid1J
      • kindly email <peter.yim@cim3.com> if you have any question.

Additional Resources

For the record ...

How To Join (while the session is in progress)

Conference Call Details

  • Date: Thursday, 28-Feb-2013
  • Start Time: 9:30am PST / 12:30pm EST / 6:30pm CET / 17:30 GMT/UTC
  • Expected Call Duration: ~2.0 hours
  • Dial-in:
    • Phone (US): +1 (206) 402-0100 ... (long distance cost may apply)
      • ... [ backup nbr: (415) 671-4335 ]
      • when prompted enter Conference ID: 141184#
    • Skype: joinconference (i.e. make a skype call to the contact with skypeID="joinconference") ... (generally free-of-charge, when connecting from your computer)
      • when prompted enter Conference ID: 141184#
      • Unfamiliar with how to do this on Skype? ...
        • Add the contact "joinconference" to your skype contact list first. To participate in the teleconference, make a skype call to "joinconference", then open the dial pad (see platform-specific instructions below) and enter the Conference ID: 141184# when prompted.
      • Can't find Skype Dial pad? ...
        • for Windows Skype users: Can't find Skype Dial pad? ... it's under the "Call" dropdown menu as "Show Dial pad"
        • for Linux Skype users: please note that the dial-pad is only available on v4.1 (or later; or on the earlier Skype versions 2.x,) if the dialpad button is not shown in the call window you need to press the "d" hotkey to enable it. ... (ref.)
  • Shared-screen support (VNC session), if applicable, will be started 5 minutes before the call at: http://vnc2.cim3.net:5800/
    • view-only password: "ontolog"
    • if you plan to be logging into this shared-screen option (which the speaker may be navigating), and you are not familiar with the process, please try to call in 5 minutes before the start of the session so that we can work out the connection logistics. Help on this will generally not be available once the presentation starts.
    • people behind corporate firewalls may have difficulty accessing this. If that is the case, please download the slides above (where applicable) and running them locally. The speaker(s) will prompt you to advance the slides during the talk.
  • In-session chat-room url: http://webconf.soaphub.org/conf/room/summit_20130228
    • instructions: once you got access to the page, click on the "settings" button, and identify yourself (by modifying the Name field from "anonymous" to your real name, like "JaneDoe").
    • You can indicate that you want to ask a question verbally by clicking on the "hand" button, and wait for the moderator to call on you; or, type and send your question into the chat window at the bottom of the screen.
    • thanks to the soaphub.org folks, one can now use a jabber/xmpp client (e.g. gtalk) to join this chatroom. Just add the room as a buddy - (in our case here) summit_20130228@soaphub.org ... Handy for mobile devices!
  • Discussions and Q & A:
    • Nominally, when a presentation is in progress, the moderator will mute everyone, except for the speaker.
    • To un-mute, press "*7" ... To mute, press "*6" (please mute your phone, especially if you are in a noisy surrounding, or if you are introducing noise, echoes, etc. into the conference line.)
    • we will usually save all questions and discussions till after all presentations are through. You are encouraged to jot down questions onto the chat-area in the mean time (that way, they get documented; and you might even get some answers in the interim, through the chat.)
    • During the Q&A / discussion segment (when everyone is muted), If you want to speak or have questions or remarks to make, please raise your hand (virtually) by clicking on the "hand button" (lower right) on the chat session page. You may speak when acknowledged by the session moderator (again, press "*7" on your phone to un-mute). Test your voice and introduce yourself first before proceeding with your remarks, please. (Please remember to click on the "hand button" again (to lower your hand) and press "*6" on your phone to mute yourself after you are done speaking.)
  • RSVP to peter.yim@cim3.com with your affiliation appreciated, ... or simply just by adding yourself to the "Expected Attendee" list below (if you are a member of the community already.)
  • Please note that this session may be recorded, and if so, the audio archive is expected to be made available as open content, along with the proceedings of the call to our community membership and the public at-large under our prevailing open IPR policy.

Attendees

  • Expecting:
    • ...
    • please add yourself to the list if you are a member of the Ontolog or Ontology Summit community, or, rsvp to <peter.yim@cim3.com> with your affiliation.



Retrieved from "https://ontologforum.com/index.php?title=ConferenceCall_2013_02_28&oldid=740"