Sven Van Poucke

Sven Van Poucke, MD

Ziekenhuis Oost-Limburg Genk

Schiepse Bos 6 3600 Genk Belgium

Tel: +32 89 32 5300

SKYPE/GIZMO: svenvanpoucke

http://svenvanpoucke.eurocv.eu

The design of a semantic content analysis platform for digital wound images.

1. General Description

Professionals dealing with wound patients need to make treatment decisions principally, but not solely based on their visual perception. The descriptive analysis of wounds however is still poorly standardized and rarely reproducible. Only recently, assessment and measurement of the temporal changes of wounds using images taken with commercially available digital cameras can be calibrated independently of any camera settings and independently of illumination features. With a growing demand for randomized clinical trials and increasing economical pressure on health budgets, the key requirement for optimal data sharing is standardization with agreements on types and definitions of structures, processes and formats used to access and share data. Supposing that all clinical terms used by the wound care community could be defined and could be readable, in a standardized format, any person and software could understand and use this data. Therefore, consistent and universally accepted vocabulary seems essential for documenting, describing and comparing wounds which evently could increase the knowledge about underlying diseases and therapeutic processes. Ontologies (controlled vocabularies) promise to help address many of the difficulties currently experienced in managing large image databases by distinguishing between the different semantic ``worlds . First, a Real World is defined, in which exists the objects and events that have been partially captured by various forms of media (micrographs, videos, etc.), a Media World, in which exists such representations of aspects of the real world, and a Cognitive World, in which exists man's ideas, interpretations and conceptualisations both about objects and events from the Real World but also about the portrayal of these objects and events in the Media World. Formalizations of the conceptualisations and ideas of all three worlds and of their interconnections exist in a fourth Logical World. For scientific and medical image databases, the distinction between these worlds is a vital one that an image ontology and data model must seek to capture. The storage, management, exchange and description of data in this domain present challenges to both clinicians and bioinformaticians. Despite a growth of online medical journals (that permit the inclusion of media objects), few of these resources are freely available, and those that are, are difficult to locate and are not cross-searchable. A need is arising for a free publicly available image database with well-structured searchable metadata.The increased cost of technological advancements related to digital imaging and data management has encouraged scientific communities to develop and adopt ontology-based knowledge representations to extend power in efficacy and efficiency. To the best of our knowledge, there are no other studies proposing a standardized quantitative image processing and analysis procedure based on the latest ontologic insights in wound care.

2. The Solution

The analysis of digital images of wounds, requires considerable expertise. The presented project aims to provide clinical wound care communities with tools to manage datasets of digital images related to wounds and to extract, analyse and structure the knowledge behind the clinical interpretation of these datasets. This information could be used to devise specific instructional strategies or support systems to make the acquisition and application of these skills more efficient and effective. The following article will present the development of a platform for feature analysis of digital wound images.

3. Methodologies & Objectives

With the development of the Woundontology Base, a backbone able to store images of chronic human wounds is provided to the wound care community. The database with its images are intended to be semi-open and provided with tools for comment and annotation. A database interface is designed to have optimal browser and platform compatibility. The Woundontology Base will be tested and applied in a critical user environment represented by the Woundontology Consortium, being a semi-open, international, virtual community of practice devoted to the advancements of research on non-invasive wound assessment by image analysis, ontology and knowledge acquisition (i.e. content-based visual information retrieval). The Consortium focuses on the optimisation of the quantification and qualification of changes in time and space of imaging data from chronic wounds. The database is actively curated by scientists of the Woundontology Consortium before publication. This project is open to all members of the wound care community and is accessible over the internet using Wiki-technology and a client-server architecture with a lightweight web-based interface. The Woundontology Base is developed with several objectives in mind. Some of them were already proposed in similar projects in other domains: i.e. the EC-funded ORIEL Project (Online Research Information Environment for the Life Sciences):

1. To provide the clinical wound care community with a freely accessible database and archive system for high-quality multidimensional digital images of wounds, with the supporting metadata.

2. To make anonimized images and their metadata available electronically via the internet for personal study, educational (clinical dicision support systems), commercial, medical and scientific research purposes.

3. To provide tools and interfaces to assist in the submission, discovery, downloading and visualization of such images, and making comparisons between them.

4. To provide links between these images and relevant items including clinical information, ontologies, etc.

5. To facilitate effective data representation and information visualisation through the construction of adaptive interfaces that meet the needs of individual users.

6. To provide tools to assist in annotation of wound images and the development of concept maps and ontologies related to these images.

7. To develop theoretical (mathematical, colorimetric,...) models related to the concepts, language and terminology used by the wound care community.

4. Wound Imaging

The imaging technology in medicine has been used most frequently to image deep organs within the human body. Recently, attention has focused on imaging technology for evaluation of the skin. Digital photography in medicine has been extensively reviewed by others. In wound care and dermatology in general, a variety of imaging techniques are currently being used including photography, surface microscopy, ultrasound, laser doppler perfusion imaging, confocal microscopy and magnetic resonance imaging. Although many of these techniques are still under research, they are showing promise in clinical dermatology to study skin properties, such as color, hydration and texture. It seems generally accepted, albeit with an understanding for the legal underpinnings of their use, that these modalities can provide additional information and can assist in the management of skin problems. Frequently, they form a part of the medical record, for which words provide no complete substitute. The value of digital image information depends upon how easily it can be located, searched for relevance, and retrieved. . As high-resolution digital cameras are obtainable within acceptable price ranges, a full understanding of cutaneous imaging techniques continue to deserve attention in a professional world where the traditional approach to skin diagnosis was long based on visible inspection only. The key to extracting more information from wound images is advanced, (semi- automated) image processing and analysis techniques. These techniques frequently offer the promise of automation, objectivity and reproducibility and have potentially higher sensitivity than exert human observers. Digital images of wounds are semantic instruments for capturing aspects of the real world. Consequently, novel bio-informatical technologies are necessary (i.e. ontologies) to support, test and optimize these new approaches. Descriptive data on images (such as a radiologists' protocol) seems essential because clinicians routinely report fewer features in a case than they subsequently agree are present. The advances in technology have also been published for patient self-examination, dermatology education and tele-dermatology. Keeping the theory behind color perception with its variations in mind, , it is quite interesting to observe that in a era of considerabel pressure on economical resources for health care, concepts such as wound bed color (red-yellow-black wound classification system), their indication of a phase in healing and their underlying organic meaning (the nonuniform mixture of black necrotic eschar, yellow necrosis and fibrin (slough), and red granulation tissue, ...), continue to be the cornerstone of clinical guidelines and protocols published by international societies and key opinion leaders without a semantic, ontologic or colorimetric formal description, definition or consensus of the used concepts or terminology. Moreover, wound bed related features assessed clinically do not aways appear to be quantitatively linked to any currently measurable biophysical parameter. What is the coherence between signs and symptoms in this context?

• Image Data

A digital image is a representation of a two-dimensional image as a finite set of digital values, called picture elements or pixels. Digital photography uses a filmless digital camera that captures an image and can store it in an electronic memory. Much of today's color imaging is based on a trichromatic pixelwise representation of color in color spaces Some used hue, saturation, and intensity measurements as a technique for color analysis of chronic wounds on the skin being closer to human perception of color but scientifically less correct. The ability to store an image in a digital format allows it to be easily displayed on a computer monitor, accurately reproduced, transferred to other storage media, printed on paper or transmitted to other computers via the internet. This versatility has made it ideal for recording and storing images of the skin. Modern digital cameras with resolution capabilities exceeding 1536 × 1024 pixels are widely available and affordable, and can provide good-quality images of the skin. While this image resolution is yet to match the resolution of 4096 × 2736 pixels achievable with 'gold standard' 35 mm film photography, it is still sufficient for routine clinical use given that a minimum resolution of 768 × 512 pixels, on a standard 17-inch monitor, has been shown to be adequate for recognition of the important features of skin lesions when viewed as digital images. Wound conditions can be monitored chronologically, with serial images superimposed by the computer to allow more accurate assessment of any alterations in the progress of the condition or responses to treatment.

In the Woundontology Base, we define an image as a 3-dimensional (2 in space, 1 in time) structure with time dimension being continuous or discrete. Many image file formats use data compression to reduce file size and save storage space. Image files can be compressed in two ways: lossless and lossy. Both cameras and computer programs allow the user to set the level of compression. Some compression algorithms, such as those used in PNG and TIFF file format, are lossless, which means no information is lost when the file is saved. The JPEG file format uses a lossy compression algorithm. The greater the compression, the more information is lost, ultimately reducing image quality or detail. JPEG uses knowledge similar to the way the brain and eyes perceive color to make this loss of detail less noticeable. In the Woundontology Base, digital images of a human wound with a reference chart (the Mac- Beth ColorChecker Chart Mini [MBCCC] (GretagMac- Beth AG, Regensdorf, Switzerland) as described by are uploaded as PNG or TIFF file with a minimal resolution of 3 megapixels. The illumination of the reference chart should be homogeneous over the field of view. As complex as the data of digital images might seem, it is uninterpretable without the associated metadata, describing key feature of images.

5. Design

• Imaging Workflow Paradigm

The Woundontology Consortium a semi-open, international, virtual community of practice aims to make a leap forward in the technologies of research in non-invasive wound assessment by image analysis, ontology and knowledge acquisition. Modelling is more than just an exercise in definitions and semantics, it basically provides a mechanism for communicating data. Invariably, complex datasets must be analysed by multiple software tools..

The development method in an semi-open environment harnesses the power of distributed peer review and transparency of process. The promise of this approach is a better quality, higher reliability, more flexibility and lower cost. (Figure 1: The Woundontology imaging workflow) The workflow is initiated when a digital image of a human wound with the MBCCC reference chart is uploaded to the Wound Image Base and is followed by the calibration of this image. Stored images are consequently interpreted by the Wound Image Visual Diagnostic Expert Group using annotation technology. Simultaneously, a domain ontology for wound images is developed by ontology experts using an ontology editor and is based on concept maps (CMs) generated during the knowledge elicitation process. The concept maps are developed by clinical experts in the wound care community. Another team, the Wound Image Color, Pattern, Texture and Shape Group (WICPTSG) explores the annotations produced by the Wound Image Visual Diagnostic Expert Group (WIVDEG) and searches for quantitatively relationships between the color, shape, pattern and texture content of annotations and their linguistic content. The tools needed to turn full circle in this imaging workflow are presented.

The Wound Image Base Group (WIBG)

Advances in storage media along with the semantic web enable us to store and distribute photographic images worldwide. The development of an online wound image management application, the terms of service, the navigation manual, the copyright and privacy policy, etc are under the directory of the Wound Image Base Group. The image base can be reached via the internet (URL: http://www.woundontology.com; login and password: wound). Initially, the WIBG will be focused on the deployment of a searchable repository, up- and downloading features of in-vivo digital wound images with calibration chart for automatic and manual calibration of images. A major incentive for wound professionals to upload their images is the ability for calibration and storage. In the future, more automated tools for interpretation of wound images will be developed.

The Wound Image Visual Diagnostic Expert Group (WIVDEG)

Photo annotation is considered a resource-intensive task, yet increasingly essential as image archives grow in size. There is an inherent conflict in the process of describing and archiving personal interpretation. Casual users are generally unwilling to expend large amounts of effort on creating the annotations which are required to organise their collections in a way they can make the best use of them. During the annotation process, the user can define new terms for each enumeration while interpreting images. This process results in a linkage between phenotypes (``observed quality of) and the image region of interest (ROI). In order to make progress, it may be essential to have large databases including challenging images, in which labels are made publically available. These labels should provide information about the object classes (ROI i.e. necrosis) present in each image, as well as their shape, pattern, texture, colorimetric content and location which will be analysed by the WICTSG. In the future, this data can be used for testing (comparing algorithms), as well as for training using supervised learning techniques.

The Wound Image Color, Texture and Shape Group (WICTSG)

In recent years, there has been a growth in mathematical models designed to explain and extend the understanding of wound healing. Multiple algorithms to analyse colors, patterns, textures and shapes of all sorts of objects have been published in the past.All with the same goal in mind: disambiguation faced during interpretation of and communication on images. Problems related to the photographical analysis of wounds can be partially solved with the current insights used in geo-spacial sciences and satellite imaging. In this project, we propose the use of a standardized protocol for extracting reliable color, texture and shape features from ROIs. The primary goal of pattern recognition is supervised or unsupervised classification. Among the various frameworks in which pattern recognition has been formulated, the statistical approach and more recently neural network techniques (i.e. Fuzzy-ARTMAP neural network), have been intensively studied and used in practice. By implementing multiscale analysis, an image is analyzed at different resolutions, revealing different characteristic at each resolution. At higher resolutions, finer texture features, as well as noise are visible. We are interested in identifying a particular range of scales where we have regions of diagnostic interest. The link with ontology in not new, with contextdriven re-adjustments of the degrees of confidence of detected labels, image analysis is improved utilizing an ontology infrastructure. One of the initial tasks to be assigned by the WICPTSG is the extraction of available algorithms for color, shape and texture analysis and the identification strengths and weaknesses of the algorithms for the domain of chronic wound images.

The Wound Image Ontology Group (WIOG)

The Cmap Server Environment

Usually terminology is gathered and organised into a taxonomy, from which key concepts are defined and related to create a concrete ontology. Domain experts are geographically distributed, the structure of the ontology is constantly evolving, and promotion for collaboration and communication among domain experts is necessary. During the knowledge elicitation process of the interpretation of wound images, knowledge should be captured and shared by supporting collaboration in distributed (decentralised) environments. This is the stage where the person managing the development of the ontology gathers, in the form of concepts and relationships between concepts, what the domain expert understands to exist in that domain. To do this, we propose the use of concept maps (CMs) as a simple graphical representation in which instances and classes are presented as nodes, and relationships between them are shown as arcs. CMs have a simple semantic that appears to be an intuitive form by which domain experts can convey their understanding of a domain. We exploit this feature in order to perform the informal modelling stage of building an ontology. The CmapTools is developed by the Florida Institute for Human & Machine Cognition, a not-for-profit research institute of the Florida University System (URL: http://cmap.ihmc.us/).

6. The Ontology for Wound Images

The current literature in artificial intelligence provides a number of interesting frameworks for developing, deploying, testing and embedding ontologies. In addition to domain-specific ontologies, image databases require a generic image ontology describing the structural content and media representations of the images themselves. There is not one single methodology for developing ontologies, nor is there a single correct result. Developing an ontology is usually an iterative process. You start with a rough first pass at the ontology followed by a revision and refining of the evolving ontology and filling in the details. In practical terms, developing an ontology includes: defining classes in the ontology, arranging the classes in a subclass-superclass hierarchy, defining slots and describing allowed values for these slots and filling in the values for slots for instances. Concepts in the ontology should be close to objects (physical or logical) and relationships in the domain of interest. These are most likely to be nouns (objects) or verbs (relationships) in sentences that describe the domain of interest. Ontologies can be built using ontology editors. What do ontologies offer over keywords? In photo collections indexed with keywords, a small subset of the controlled keyword set is associated with an image. The keywords themselves are unrelated atoms. If we consider the terms of the ontology to be our controlled keyword list, using an ontology and a structured description based on this ontology changes the annotation and querying process in a number of ways: Firstly, it guides the annotation process using restrictions and default information. Secondly, it makes the relation between property values and agents explicit, telling which property value is connected using which property to which element of the subject matter or the photo itself. Consider "necrosis under large scab." Reduced to keywords, "large" can refer to the necrosis, the scab, or even the photo. moreover, an ontology provides relations between the terms. It is of critical importance to keep in mind that an ontology is never perfect nor complete. It is an abstraction of a particular domain and is determined by the use the ontology will be put.

7. Discussion

There is a consensus within the wound care community that a systemic approach to the patient's assessment is necessary to treat a chronic wound ("Look at the whole patient, not just the hole in the patient."). Therefore, digital imaging of wounds constitutes only a small piece of the assessment process. During the assessment of wounds, the experience of the clinician plays a significant role in identifying the actual state of a wound. The assessment is caried out visually and qualitatively based on his-her subjective experience. Therefore, this procedure suffers from potential interpretational variability, lack of comparative analysis, and it is time consuming. Life and disease in their full complexity are processes. An interpretation of a digital wound image at time x, is not enough to simply catalogue the properties of biological systems. In the past, the question of what a process is was left to the philosophers (Aristotle's entelechy). Confronting interpretations of clinical images from individual patients with generalized concepts of disease should recognize the limitations related to differences between the instantiated anatomy supporting applications of biomedical knowledge in clinical care and in fields such as image analysis and the canonical anatomy being the field of anatomy that comprises the synthesis of generalizations based on anatomical observations that describe idealized anatomy.

It is the Consortium´s opinion that exploiting the complete potential of the internet to develop novel technologies for integration of large, complex information resources can mean a step foreward in a more efficient data flow during the assessment and treatment of wounds. Future trends in the development of this semantic content analysis platform might include the integration of WordNet® (the lexical database of English, developed by Cognitive Science Laboratory Princeton University) in the annotation process, the translation of this model to other disciplines in medicine using imaging techniques and cross-granularity integration. With the growing importance of preventive medicine, outpatient follow-up of chronic wounds could be managed using mobile phone camera's with a connection to our server. The knowledge elicitation proces and ontology development could eventually lead to (semi-)automatic wound image analysis and could aid in clinical decision support systems and education.

8. Conclusion

The technology presented here should be able to provide a more automated and standardized wound quantification that meet the needs of clinical professionals, imaging centers, and large enterprises conducting multi-site clinical trials. We have focused our discussion on the deployment of applications for imaging wounds, but the points made are applicable to all forms of large-scale multidimensional data acquisition. The defined structure of data models and their physical implementations must be shared as open as possible, to enable scientists and clinicians to explore new approaches and methods of analysis. The mission of the Woundontology Consortium is to foster cooperative efforts to translate academic research into services for the benefit of society, support of the broader research and education. Only with joined efforts from the wound care community, system biology community and ontology community this project is viable

9. Current Status

We are now entered in the recruitment phase of this project. The process of finding the right experts and professionals in each group will be achieved using scientific publications, professional societies and internet communications. The individual privileges and functions need to be defined by the group directors. Inter-personal communication for this is promoted by e-mail, the consortium WIKI, discussion group (URL: http://groups.google.com/group/woundontology; http://www.knoodl.com/ui/groups/woundontology) and videoconferencing using Skype (URL: http://www.skype.com) and GIZMO (URL: http://www.gizmoproject.com/). Domain experts (chronic wounds) and ontology scientists interested in this project can subscribe to one of the discussion groups