Common Information Model Primer

Chapter 9: Transforming Data to CIM

    Learning Objectives

    • Understand how the concept of bi-directional transformation is applied
    • Understand the difference between stateless and stateful

      Bi-directional Transformation

      Section 8 provided a simple example of how existing internal data structures can be mapped to the CIM. Given that CIM-derived profiles define interface standards, not internal data structures, it is very common for existing applications to map their internal data to and from the CIM.

      The transformation is one-way when the requirement is only to either import or export data. But for many applications there is a need to both import and export to CIM and to ensure that there is consistency both ways. Consistency checking will ensure that a native data element within the application is always mapped to the same CIM element and vice-versa.

      Such bi-directional transformation is essential when an application is to participate in an enterprise environment where it is communicating with other applications only via CIM-based messaging. There are two primary means of implementing such a transformation depending upon the structure and format of the source data: stateless or stateful.

      Stateless vs. Stateful Transformation

      In a stateless transformation all the information required to go from A to B and from B to A is contained within the data itself or can be automatically calculated /extracted without any external input. For applications with internal data structures that are derived from the CIM, it is likely that transformations to and from the CIM will be stateless as there will be one-to-one mappings between the CIM and internal application data structures.

      In a stateful transformation either the source or target does not contain sufficient data to enable a bi-directional transformation without loss of data. In such circumstances any transformation must produce, along with the output, additional state data that will allow any subsequent reverse transformation to correctly map into the original source data.

      When defining CIM transformations the most common issue that arises in creating a stateless transformation is ensuring that the CIM mRID attribute can be mapped into the application’s source data. In many legacy systems the unique identity of a piece of equipment or other data element is a simple database table entry key. When mapping a CIM element that is uniquely identified by a 128 bit UUID there may be no field within the application’s internal data structure to store this identifier. As such any subsequent re-export of the data back into CIM would not have this same unique identity.

      The solution is to store additional mapping state data that maps the original identifier to the identifier in the system. Transforms therefore need to know both the source data and any state data about previous transformations. This same approach may also be required when multiple components in the CIM maps to a single entry in the application data (or vice-versa), ensuring that the same components are mapped in all transformations.

      To the end-user this process should be seamless, but for developers and vendors there is an order of magnitude increase in complexity when writing and maintaining stateful transformations rather than stateless transforms.

      Model-Driven Transformation

      In Section 2 the concept of distinguishing between structure and format was briefly mentioned, and it is this concept that enables Model Driven Engineering (MDE) and Model Driven Architectures (MDA) to be applied with Model Driven Transformation (MDT). With MDT the transformations are written at the structural level rather than format-to-format.

      For example, if the source data is a database and the target data is an XML file with MDT the source and target serialization formats are not known within the transform, it instead defines itself against the structure of the data. This enables the same data, defined by the same structure to be serialized in a number of formats, whether it be a database, flat file, XML, network data stream, and so on, as shown in Figure 9-1.

      Figure 9-1: Multiple Serialization Formats/Locations

      Figure 9-1: Multiple Serialization Formats/Locations

      The format-agnostic data objects are identical in structure no matter what the source of the data is and so the transform is written to operate against these data objects rather than to read/write to database tables or parse file formats. Applications are written to process the data objects rather than requiring multiple interfaces to deal with the different possible sources. This approach also modularizes the software, separating the parsing of data from a file or reading from a database from the application logic itself.

      Figure 9-2: Model Driven Transformation Process

      Figure 9-2: Model Driven Transformation Process

      This means the transformation is just one part of the overall process with multiple stages as shown in Figure 9-2. The data objects may be defined by a complex inheritance structure like the CIM or be a simple flat structure derived from a file format.

      The other advantage of this approach is that data transformation itself is written against the structural elements making its syntax clearer, more concise and easier to read than code design to convert both format and structure simultaneously. An example of a MDT language, QVTO (Query/View/Transform Operational)41 is shown below:

      mapping CIM::IEC61970::Wires::ACLineSegment:: aclToBranch() : PF::Branch{	
          var bus1 : PF::Bus := self.Terminals->asSequence()->first().
                    	        TopologicalNode.map nodeToBus();
        	    var bus2 : PF::Bus := self.Terminals->asSequence()->last().
                    	        TopologicalNode.map nodeToBus();
        	    if (bus1.number < bus2.number) then{
         		        FromBus := bus1;
         		        ToBus := bus2;
         		        MeteredBus := bus1;
        	    }else{
         		        FromBus := bus2;
         		        ToBus := bus1;
         		        MeteredBus := bus2;	
        	    }endif;
      	
        	    circuit := self.name.substring(0,2);
        	    R := self.r.toPerUnit(self.BaseVoltage.nominalVoltage);
        	    X := self.x.toPerUnit(self.BaseVoltage.nominalVoltage);
        	    B := self.bch.toPerUnit(self.BaseVoltage.nominalVoltage);
        	    length := self.length;
      }
      

      This example code is for converting CIM ACLineSegment to a proprietary power-flow application’s Branch. The mapping function defines its input using the full CIM package structure and the class for its output is Branch under the PF package. The attributes RX, and B in the power-flow format are assigned by taking the corresponding CIM values rx, and bch attributes then passing them through a helper function that converts them to per-unit using their value and that of the line segment’s voltage (obtained via the segment’s BaseVoltage association).

      Mapping functions may call other mapping functions so that, in this example, the From and To buses are assigned by taking the first and last Terminal on the ACLineSegment, finding their TopologicalNode’s then mapping them to the power-flow format’s Bus using another mapping function, nodeToBus().

      What should be taken from this example is that the code is mapping the data based on its structure and relationships. At this level it is not known if the CIM data is coming from RDF XML, a CIM XSD Message, a database or even a custom CIM binary format. Additionally, nothing is known about the power-flow application’s file format or whether the resulting data is intended to be written to a file, sent across a network or be used with an API.

      Footnotes

      41 QVT (Operational) is an Object Management Group specification for model to model transformation with an open-source Eclipse implementation available that runs on top of Java. The OMG specification can be found at https://www.omg.org/spec/QVT/.

      Example: CIM to PSS/E

      An example of how MDT can be used is taking a real-world example of translating CIM RDF XML data into PSS/E RAW42 format43 and looks at the high-level process involved in creating a CIM to PSS/E transform using MDT. PSS/E RAW is a space/comma delimited ASCII file format for bus-branch, balanced network data and, unlike the CIM, is not derived from an overall UML model (or if it is, the UML is not publicly available). As such, to create a CIM to PSS/E converter using MDT a number of steps are involved:

      1. The PSS/E data structure must be derived to create a PSS/E meta-model
      2. A parser for PSS/E RAW files must be written to create the format-agnostic data objects
      3. A transform between the CIM classes and the PSS/E meta-model elements
      4. A CIM to RDF XML serialization interface must be written to create the CIM RDF XML from the CIM data objects.

      It is assumed that the CIM meta-model will be the UML (or a subset of the UML from the appropriate profile group).

      Defining the PSS/E Meta-Model

      Defining the PSS/E meta-model involves a reverse engineering of the format to determine the elements, attribute and associations. While some formats may provide a mechanism to automatically determine this meta-data (that is, extracting a database schema) with PSS/E the definition is primarily a manual task based on the comments and meta-data within the instance files (and if available, any documentation). With PSS/E there are a number of complex elements including: Bus, Branch, Transformer, Load, Generator and Shunt, as well as containment elements Area and Zone and the Owner element.

      Unlike CIM there is no explicit inheritance so the direct mirror of the file format is to have a non-hierarchical class structure. Often during the creation of a meta-model, however, patterns emerge between classes (that is, multiple classes have a single association to a Bus or contain a name or ID attribute) and so a class hierarchy may be introduced to make the meta-model more elegant and remove duplication.

      Associations in PSS/E are defined as index references to other classes (that is, a Load has a Bus index 1001 which corresponds to a Bus entry with ID 1001). In the meta-model these are instead defined as associations so that when the data is read into the format-agnostic objects the index references are resolved and become direct, in-memory associations (that is, pointers).

      Parsing/Writing PSS/E RAW

      The parsing of the files into memory is a case of identifying the format of the files then writing software that will read this format, creating the appropriate in-memory data object instances for each element from the processed data. Since PSS/E is an ASCII format a parser can be written in most programming languages.

      The processing is generally a two-step process, creating all the objects and their attributes then resolving all the index references (since some elements refer to elements that are at the end of the file). Processing even large network models in this format can be accomplished very quickly.

      Writing PSS/E data involves the processing of the in-memory data objects and serializing them to a file in the correct order and format. When the structure is derived from the file format serialization can be a relatively straightforward process; looping through each element and outputting the attributes in order.

      More complex serialization rules are required to deal with associations (that is, knowing the correct index attribute to refer to) or dealing with fixed enumerations where the meta-model has human-readable names such as OutOfService or InService where the file is expecting the enumeration value of 0 or 1.

      Transforming CIM to PSS/E

      With the PSS/E meta-model defined and the CIM meta-model taken from the UML the transform is written against these meta-models. In some cases, there is simple one-to-one mapping, but in others more complex logic is required. Some examples of the transformation logic are shown below in QVTO (with well documented code to explain what is happening).

      CIM EnergyConsumer to PSS/E Load

      A relatively straightforward one-to-one mapping is that of the CIM’s EnergyConsumer to PSS/E Load as shown below.

      Mapping CIM::IEC61970::Wires::EnergyConsumer:: getLoad() : PSSE::Load {
        -- Set the Bus to be the first (and only)
          -- Terminal’s Topological Node mapped to a PSSE:Bus
          result.Bus := self.Terminals->first().TopologicalNode.map getBus();
          result.id := self.name;
        
          var p : Real := 0.0;
          var q : Real := 0.0;
        
          -- If the EnergyConsumer’s Terminal has an SvPowerFlow associated
          -- with it then we use the values from it, otherwise we use the fixed
          -- real and reactive power values for the load
          if (self.Terminals->first().SvPowerFlow <> null) then{
             p := self.Terminals->first().SvPowerFlow.p;
             q := self.Terminals->first().SvPowerFlow.q;
          }else{
             if (self.isSet(“pfixed”)) then{ p := self.pfixed;}endif;
             if (self.isSet(“qfixed”)) then{ q := self.qfixed;}endif;
          }endif;
        
          -- If a LoadResponse is set then this load will have Constant, Current
          -- and Impedance components. The PSS/E values are thus set based on the
          -- total real/reactive power multiplied by the component
          -- (which should add to 1.0)
          if (self.LoadResponse <> null) then{
             if (not self.LoadResponse.exponentModel) then{
                 Pmva := p * self.LoadResponse.pConstantPower;
                 Qmva := q * self.LoadResponse.qConstantPower;
                 Pcurrent := p * self.LoadResponse.pConstantCurrent;
                 Qcurrent := q * self.LoadResponse.qConstantCurrent;
                 Padmittance := p * self.LoadResponse.pConstantImpedance;
                 Qadmittance := q * self.LoadResponse.qConstantImpedance;     
             }else{
                 -- PSS/E cannot deal with the exponent model so set all as MVA 
                 Pmva := p;
                 Qmva := q;
           }endif;
          }else{
             -- If no LoadResponse is present then only the constant power is set
             Pmva := p;
             Qmva := q;   
          }endif;
        
        
        -- The PSS/E Area is mapped to the SubGeographicalRegion and the Zone to
          -- the GeographicalRegion
          result.Area := self.getSubGeographicalRegion().map getArea();
          result.Zone := self.getGeographicalRegion().map getZone();
        
          -- For Load’s with an OperatingShare the Owner becomes its OperatingParticipant
          -- ignoring any but the first as PSS/E supports only a single owner
          result.Owner := self.OperatingShare->first().
                            OperatingParticipant.map getOwner();
        
          -- The isConnected() helper function checks if the Terminal is connected
          -- and so sets the status  oolean accordingly
          status := self.isConnected();
      }
      

      As can be seen in this example, a number of other mapping functions are called to convert TopologicalNode to Bus and GeograpicalRegion to Zone amongst others. The association paths can be followed with a simple syntax so that the EnergyConsumer’s (self in this code as it is the source of the mapping) LoadResponseCharacteristic can be accessed via its LoadReponse association with the simple syntax self.LoadReponse with the internal LoadResponseCharacteristic attributes then accessed by appending them as self.LoadResponse.pConstantCurrent or self.LoadReponse.exponentModel.

      SynchronousMachine to Generator

      The mapping of generation is more complex as it as the CIM’s SynchronousMachine class may not be behaving as a generator so there are some initial conditions placed. This mapping is also a one-to-many as it may produce a PSS/E Generator and multiple PSS/E Owner Share elements.

      Mapping CIM::IEC61970::Wires::SynchronousMachine:: getGen() : PSSE::Generator
          -- Only SynchronousMachines operating as a Generator with a GeneratingUnit
          -- or as a condenser without a GeneratingUnit are mapped to the PSS/E Generator
          when {
             (self.GeneratingUnit <> null and self.operatingMode = 
                  CIM::IEC61970::Wires::SynchronousMachineOperatingMode::generator) or
             (self.GeneratingUnit = null and self.operatingMode = 
                  CIM::IEC61970::Wires::SynchronousMachineOperatingMode::condenser)}{
        
          -- The Generator’s Bus is mapped to the SynchronousMachine’s Terminal’s
          -- TopologicalNode
          result.Bus := self.Terminals->first().TopologicalNode.map getBus();
          -- The PSS/E ID is mapped to the name 
          result.id := self.name;
        
        
      -- If the SynchronousMachine has a RegulatingControl that controls Voltage
          -- The the regulating bus for the Generator must be set to the
          -- Bus the RegulatingControl is controlling (obtained via the TopologicalNode
          -- of the Terminal it is associated to) and the setpoint set to the
          -- RegulatingControl’s regulated set-point
          if (self.RegulatingControl <> null and self.RegulatingControl.mode = 
                CIM::IEC61970::Wires::RegulatingControlModeKind::voltage) then{
             var cBus := self.RegulatingControl.Terminal.TopologicalNode.map getBus();
             result.VregulatedSetpoint := self.RegulatingControl.targetValue/cBus.baseKV;
             if (cBus <> null and cBus <> result.Bus) then{
                 result.Regulating := cBus;
             }endif;
          }endif;
        
          -- If a generator the Pgen, max and min values are taken from the GeneratingUnit
          if (self.GeneratingUnit <> null) then{
             Pgen := self.GeneratingUnit.nominalP;
             Pmax := self.GeneratingUnit.maxOperatingP;
             Pmin := self.GeneratingUnit.minOperatingP;
          }endif;
        
          -- If the ratedS is set that is taken as the base power for per-unit conversion
          -- otherwise the system base is used
          var mBase := sBase;
          if (self.isSet(“ratedS”)) then{ mBase := self.ratedS;}endif;
          MVAbase := mBase;
          -- The reactive power gen, max and min are taken from the SynchronousMachine
          Qgen := self.baseQ;
          Qmax := self.maxQ;
          Qmin := self.minQ;
          Rmachine := zToPerUnit(self.r, result.Bus.baseKV, mBase);
          Xmachine := zToPerUnit(self.x, result.Bus.baseKV, mBase);
        
          -- If the Terminal has an SvPowerFlow this is taken as being the actual
          -- power output of the machine. The values are multiplied by -1 as CIM’s
          -- representation of powerflow has a Load as positive and a Source as negative
          -- (in terms of direction of flow into or from the bus)
          if (self.Terminals->first().SvPowerFlow <> null) then{
             Pgen := self.Terminals->first().SvPowerFlow.p * -1;
             Qgen := self.Terminals->first().SvPowerFlow.q * -1;
          }endif;
        
          -- A PSS/E OwnerShare is created for each OperatingParticipant in the CIM.
          -- As such this mapping produces not only a PSS/E Generator but potentially
          -- multiple PSS/E OwnerShare elements
          self.OperatingShare->forEach(op){
             var share := object PSSE::OwnerShare{};
             share.Owner := op.OperatingParticipant.map getOwner();
             share.fraction := op.percentage/100;
             result.Owners := share;
          };
        
          -- The isConnected() helper function checks if the Terminal is connected
          -- and so sets the status  oolean accordingly
          result.status := self.isConnected();
      }
      

      This and the EnergyConsumer-Load mapping show that the mapping of the actual electrical parameters (load, generation) often form the minority of the transformation code. Dealing with the differences in structure can account for the majority of the effort and it is here that the code is both easier to write and maintain when it is written against the structure rather than trying to map CIM XML directly to PSS/E RAW in a single function.

      Footnotes

      42 PSS/E is a power system analysis tool from Siemens PTI that is widely used in the power industry. The PSS/E RAW format is its native file format.

      43 A.W. McMorran, G.W. Ault, I.M. Elders, C.E.T. Foote, G.M. Burt, J.R. McDonald, “Translating CIM XML Power System Data to a Proprietary Format for System Simulation”, IEEE Transactions on Power Systems, February 2004, Volume 19, Number 1, pp. 229-235.

      Benefits of a Model Driven Architecture

      An MDA approach when dealing with CIM data offers a number of benefits. By ensuring that all data has a defined structure with a meta-model, users can understand and identify how it maps to the CIM. Model Driven Transformation makes it easier to write and maintain the mapping of data between systems even though, on the surface, the process appears more complex. By separating the parsing of the data from the transformation itself the process is modularized, with the parsing module re-usable in a number of processes and simplifying the transformations so they do not have to deal with the added complexity of file formats or database access. The example shown here contains parts of a CIM to PSS/E transform to show a real-world example of how MDT can be used with CIM data and how a meta-model can be derived from existing, proprietary formats to support this architecture.

      Case Study

      Jeff Kimble has learned that the CIM, because it is a model, can support model driven transformation. This is useful information because his organization was considering acquiring a work management system that stated that some of its internal structures were CIM-derived. One of the systems integrators that Electric Innovation Utility was considering using to integrate this system with other necessary applications such as GIS and CIS have been pushing to use their proprietary adapters for the job.

      Jeff realizes that since the work management application is CIM-derived, if they support CIM adapters at the enterprise service bus, they could potentially have a stateless bi-directional transformation. This will save a tremendous amount of effort creating the adapters. Business logic will not need to be created to capture application state. Information will be preserved as it passes from system to system, without extra coding in the adapters.

      If the internal data of an application is derived from the CIM, a bi-directional transformation is most likely to be:

      A

      Stateless

      B

      Stateful

      C

      Both stateful and stateless because it is bi-directional

      D

      Neither stateful nor stateless

      A. Stateless

      Which response most closely describes a stateful transformation?

      A

      A stateful transformation has a token that can be used to pass application borders.

      B

      The source or target does not contain sufficient data to enable a bi-directional transformation without loss of data.

      C

      The source or target contains enough information for bi-directional transformation without loss of data.

      D

      A stateful transformation is not concerned with transformation.

      B. The source or target does not contain sufficient data to enable a bi-directional transformation without loss of data.

      In Model Driven Transformation (MDT), the code is:

      A

      Based on the CIM or other UML.

      B

      Based on XML.

      C

      Based on a database.

      D

      The data based on its structure and relationships, not on any particular model, database, or file.

      D. The data based on its structure and relationships, not on any particular model, database, or file.

      Model Driven Transformation makes it easier to write and maintain the mapping of data between systems when:

      A

      The data is based on the CIM.

      B

      The data is unidirectional.

      C

      The data has a defined structure with a meta-model.

      D

      The data is bi-directional.

      C. The data has a defined structure with a meta-model.

      By keeping the data transformation modular, there is the benefit that:

      A

      It is re-usable in a number of processes.

      B

      The transformation deals with the added complexity of database access and file formats.

      C

      It isn’t based on real-world examples.

      D

      The data is unidirectional.

      A. It is re-usable in a number of processes.

      We use cookies to improve your experience on our website. By continuing to use this website, you agree to the use of cookies. To learn more about how we use cookies, please see our Cookie Policy.