New:

26 February 2011: Questionnaire for Use-Cases: Problem-Solution-Metrics Matrix

Collecting Use-Cases - Connecting to Intended Audiences - Measuring Value

• Problem-Solution-Metrics Matrix
  • Text Questionnaire below should be copied and pasted at end of page for each new Use-Case. (Note: OAF stands for Ontology Application Framework--from Track 1.)
  • Graphic Matrix (shown in following image) will be compiled from Questionnaire with each Use-Case becoming a row in the table. Please Specify Metrics Carefully. To be published as pdf.

Use-Case shown above corresponds to the first entry in the Questionnaire for Use-Cases below.

Track 1 Ontology Application Framework -- Application Categories

It is requested that your Use-Case Name be accompanied by an Application Category from the four detailed in the Ontology Application Framework from Track 1:

• Integration
• Decision Support
• Semantic Augmentation
• Knowledge Management

Please Note that Track 3 Value Metrics-focused Use-Cases are condensed and meant to be gathered together in tabular form to highlight Metrics. While it will include material from Track 2 Case Studies, it is not intended to be a Full Use Case. For that we provide links to those full Use-Cases

Connecting to the Full Track 2 Case Studies

• Natural Language Understanding.
• Integration of Multiple Systems from Multiple Companies.
• [[OntologySummit2011_ApplicationCases_Synthesis|OntologySummit2011: Application and Use Cases synthesis (2LKI)

Case Study Summaries Each Case Study participant was asked to provide a grid on one slide, outlining the business problem, the solution, key features (or screen shot) and business benefits. The aim of this was to be able to identify what sort of "Ontology" this was in terms of the application framework once this was completed, and what metrics (if any) were avilable to determine the business benefits. (2OR6)

Each Case Study (2OR7)

* Challenge    (2OR8)
o What business problem the Ontology set out to address    (2OR9)
* Solution    (2ORA)
o What we mean by ontology in this case e.g. application, conceptual model    (2ORB)
* Screen shot or key features    (2ORC)
o Give a flavor of the ontology    (2ORD)
* Benefits    (2ORE)
o What metrics if any were used to demonstrate the benefits of this ontology.    (2ORF) 

Integration of Multiple Systems from Multiple Companies YefimZhuk, Sallie Mae (2OT1)

Challenge (2ORH)

* Multiple systems and sources of knowledge in different parts of the enterprise, owned by different communities of practice.    (2ORI)
* Gaining time and commitment from subject matter experts to ensure completeness of the model.    (2ORJ)
* Different groups see different shades of meaning and application for similar terms, in different contexts.    (2ORK)
* Needs a unifying approach supporting local views    (2ORL) 

Key Ontology Features (2ORM)

Solution (2ORN)

* Facilitation of knowledge gathering using ontology engineering methods.    (2ORO)
* Formal ontology notation for single ontology, while presenting views and facets of this to subject matter experts.    (2ORP)
* Curation of the ontology    (2ORQ) 

Benefits (2ORR)

* Best use of subject matter experts' time and resources    (2ORS)
* Curatorship of Enterprise Semantic Architect ensures quality, consistency and completeness of the ontology    (2ORT)
* Collaboration in industry standardization efforts (e.g. EDM Council), via common semantics    (2ORU)
* Ensures that the knowledge captured at Sallie Mae is taken forward to industry-wide standardization efforts which we can then use    (2ORV) 

Standardization of Terms and Definitions for Financial Services]].

Specifying Metrics in the Use-Case

Please specify both Quantitative and Qualitative Metrics. Such Metrics should include, but are not restricted to metrics cited in Track 1 Ontology Application Framework:

• Quality,
• Performance,
• Maintenance,
• Cost,
• Portability,
• Reliability,
• Scalability,
• Robustness,
• Usability, and
• Extensibility.

If using existing Use-Cases please give Metrics cited by the authors of the Use-Case if available.

If possible be specific, such as ROI, and explain or define the term for the Metric if not well known.

16 February 2011: Initial Community Input Page

• OntologySummit2011: Track-3: Value Metrics, Value Models and the Value Proposition - Community Input (maintained by ToddSchneider & Rex Brooks & ??)
  • Value Metrics
    • Expectations Management: How do we manage expectations?--Todd and Rex will monitor and add material from chats, but request direct community input (please edit wiki to add your thoughts):
      • After a couple of weeks of looking for well documented Use-Cases with clear benefit measurements, it seems like serious benefit measurement is more likely to occur in the implementation of IT systems than in the implementation of IT systems with semantic consistency as a goal or the use of ontology as a significant and explicit factor or feature. In fact, I have added a Use Case for Supply chain Management System as an example of documented benefits without specific semantic consistency as an explicit goal
      • This begs the question of whether there are simply insufficient Use Cases with clear benefit measurement for cases that explicitly include semantics and-or ontology --03-13-2011, Rex Brooks
      • Add your input on expectations here.
  • Add your Value Metrics Topic here
    • Add your explanation of your Value Metrics Topic here
      • Add your input here
      • Add your input here
  • Value Metrics Synthesis
    • Quantitative Measurements--Todd and Rex will monitor & add material from chats, but request community input:
      • An Ontology Development is only viable if it increases productivity by 15%. (Is this example reasonable?)
        • Add your input here
    • Qualitative Measurements--Todd and Rex will monitor & add material from chats, but request community input:
      • Add your input here.
  • Value Models
    • What models are available?
      • Customer Satisfaction Business Model
      • Actionable Business Intelligence
      • Service Orientation
      • Complex Business Events & Workflows
      • Collaborative Operations
      • Interoperable Business Services
      • Add your input (new value models) here.
    • How well do they model actual value?
      • Qualitative Market Research has wide error margin for Customer Satisfaction.
    • Do they use particular metrics?
      • Add your input here.
    • "'What metrics are appropriate per model? Inappropriate?

Questionnaire for Use-Cases: Problem-Solution-Metrics Matrix

Note: Add your Use-Case per the following instructions:

Please login, click on "Edit text of this page" at bottom of this page, then, in the editing box, copy the entire section that follows down to the line that ends immediately above the heading "Add new Use-Case Below", then paste it into the editing box just below that same heading and fill-in the information for the Use-Case you are adding. Then click on "Preview" to check your Use-Case and once satisfied that it is complete, please click "Save" with the Summary "Added (name) Use-Case."

Use-Case Name and OAF Category if appropriate:

Replace this text with Use-Case Name.

Use-Case Problem(s) & Description:

Replace this text with Use-Case Problem(s) & Description. Please be specific, concise & keep in mind that this Track is aimed at Metrics.

Use-Case Solution Used or Proposed:

Replace this text with Suggested or Expected Use-Case Solution. Please ask yourself what kind of solution is needed from the viewpoint of the Business Case?

Is an Ontology-Based Solution really called for in this Use-Case? Why or Why not?

Metrics needed to measure the success of Used or Proposed Solution to the Use-Case Problem(s):

Replace this text with specific details of metrics. What is measured? How is it measured? What does the Metric show?

What is the Intended Audience & How do we reach them:

Replace this text with the name and description of Intended Audience. Please be specific, with particular attention to the kind of language or vernacular understood by the intended audience and include opinion about appropriate means to capture attention of the audience.

What Else needs to be learned from our Use-Cases:

Replace this text with suggestions, thoughts & brief discussions for what else needs to be learned from our use-cases in addition to metrics to help us make the case for ontology.

---

Add new Use-Case Below:

(Note: Each Use-Case will be a row in a typical spreadsheet type matrix with each question comprising a column, as shown in Figure 1 above. Additional 2D matrices may be added using different, but related, questions for columns as shown in Figure 2 above.)

Use-Case Name and OAF Category if appropriate:

2006CRM-TopTierIT-ProviderFor-AutomotiveAfterMarketRetailer

Use-Case Problem(s) & Description:

Eroding Customer Loyalty & Aging Technology requires Retailer to Improve Customer Shopping Experience.

Use-Case Solution Used or Proposed:

"Open Standards (EDI-SQL-based RDBMS) and up-to-date (J2EE-based Platform) "Flexible, Web-based" IT system used for company-wide "Integration Framework."

NO Ontology-Based Solution called for in this Use-Case? Why or Why not is a guess since the example did not mention this at all, though the use of an EDI-SQL RDBMS provides a de-facto semantically consistent Datamodel.

Metrics needed to measure the success of Used or Proposed Solution to the Use-Case Problem(s):

• Quantitative Metrics:
  • 3-year Conventionally measured ROI
  • Faster Checkout measured
  • Reduced costs cited
  • 12-month Implementation
• Qualitative Metric
  • Improved Customer Loyalty

What is the Intended Audience & How do we reach them:

• Intended Audiences:
  • Automotive Aftermarket Customers
  • This Retailer's Management
• Audiences for Ontology
  • All Retailers' Management in Highly-Competitive, Public Markets

What Else needs to be learned from our Use-Cases:

The fact that some hidden semantic consistency in both the open standard terminologies used and in the terminologies of the Platform used can be cited, but the problem with this is that it is not very specific to the metrics used to measure the success of this solution.

---

Use-Case Name and OAF Category if appropriate:

Digital Music Archive (DMA) for the Norwegian National Broadcaster (NRK)

Knowledge Management

Use-Case Problem(s) & Description:

Public broadcasters have large archives ranging back 60+ years including sound assets on bakelite, vinyl and wax. Some older assets show remarkable longevity, but modern storage formats like digital video tape, certain CDs, tapes, etc are not as robust. At NRK many tapes recorded in the late 80s and early 90s could be recovered within 5 years without immediate action tor preserve these assets for the future.

Use-Case Solution Used or Proposed:

Model the Repository using semantic web technology (XML-based business rules), including transcription of metadata from well-structured, high-quality paper-based non-relational analog system to digital, semantically aligned, relational database system while completely revamping entire radio and television broadcasting production process and remastering library of recordings.

An ontology-based solution was necessary, albeit couched in Semantic Web terminology. However, with an estimated 150+ million triplestore anticipated, a semantically-aligned RDBM was implemented to scale up to a Semantic Web based publication layer for the user interface..

Metrics needed to measure the success of Used or Proposed Solution to the Use-Case Problem(s):

It continued to be tested to evaluate scalability of available systems as of 2007. Specific tests and results were not given.

Success is measured against expected benefits:

• enhanced (improved) archive access
• discovery of and navigation to hidden facts associated by metadata previously unavailable without object-based technology
• efficient, multi-channel archive access with automated ordering and production flow also previously unavailable
• enhanced (improved) metadata representation,including multiple file formats (including multimedia, images, interviews, links, etc)
• ease of integration across multiple archives and resources in future

What is the Intended Audience & How do we reach them:

Intended audience appears to be consumers of Norwegian Public Broadcasting.

What Else needs to be learned from our Use-Cases:

Of particular interest is the fact that this archive was specifically designed to be compatible with future accessing technologies, specifically SPARQL. The fact that this example failed to note the metrics used indicates that as of 2007, metrics were not deemed sufficiently important to mention.

---

Use-Case Name and OAF Category if appropriate:

Replace this text with Use-Case Name.

Use-Case Problem(s) & Description:

Replace this text with Use-Case Problem(s) & Description. Please be specific, concise & keep in mind that this Track is aimed at Metrics.

Use-Case Solution Used or Proposed:

Replace this text with Suggested or Expected Use-Case Solution. Please ask yourself what kind of solution is needed from the viewpoint of the Business Case?

Is an Ontology-Based Solution really called for in this Use-Case? Why or Why not?

Metrics needed to measure the success of Used or Proposed Solution to the Use-Case Problem(s):

Replace this text with specific details of metrics. What is measured? How is it measured? What does the Metric show?

What is the Intended Audience & How do we reach them:

Replace this text with the name and description of Intended Audience. Please be specific, with particular attention to the kind of language or vernacular understood by the intended audience and include opinion about appropriate means to capture attention of the audience.

What Else needs to be learned from our Use-Cases:

Replace this text with suggestions, thoughts & brief discussions for what else needs to be learned from our use-cases in addition to metrics to help us make the case for ontology.

---