WO2007118642A2

WO2007118642A2 - Method for checking bacnet devices for conformity, interoperability and performance

Info

Publication number: WO2007118642A2
Application number: PCT/EP2007/003188
Authority: WO
Inventors: Volker Pregizer; Frank Schubert
Original assignee: Fraport Ag Frankfurt Airport Services Worldwide; MBS Gesellschaft für Entwicklung und Handel von Software und Investitionsgütern mbH
Priority date: 2006-04-10
Filing date: 2007-04-10
Publication date: 2007-10-25
Also published as: WO2007118642A3; DE102006016760A1

Abstract

The invention relates to a method for automated checking of building automation devices for conformity to the BACnet standard, a method for checking building automation devices to BACnet standard for interoperability and a method for checking building automation device in a network to the BACnet standard for performance.

Description

Method for testing BACnet devices for conformity, interoperability and performance

The invention relates to a method for the evaluation of BACnet building automation devices according to the BACnet standard in conjunction with each other within real estate, especially in large properties such as airports and building complexes. The method includes several aspects, each of which has its own significance, in particular a method for the automated testing of building automation devices for conformity to the BACnet standard taking into account user-specific requirements, a method for testing building automation devices according to the BACnet standard for interoperability and a method for testing building automation equipment in a network according to the BACnet standard on performance. The invention also relates to a software program for carrying out the method for checking for conformity.

Building automation systems are devices, software and services for automatic control and regulation, monitoring and optimization as well as operation and management for the energy-efficient, economic and safe operation of technical building equipment. Here, a variety of different devices from different manufacturers can be networked together. In the building automation (GA) of large properties such as major airports, the use of the internationally standardized communication protocol BACnet (ANSI / ASHRAE 135-2004 or DIN EN ISO 16484-5) - hereafter referred to as "BACnet" - can be used as a communication standard for more competition among the Providers of GA facilities.

The BACnet standard defines a variety of services, objects,

Communication protocols, etc. implemented by different manufacturers to varying degrees and by the user in different Dimensions are used. For a specific property, eg a major airport, a comprehensive specification is therefore developed for the building automation equipment within a specific project, which considers all optional options of the communication protocol and dictates to the manufacturers which specifications of the BACnet standard must be fulfilled in which form.

To assess the BACnet suitability and the user-specific requirements, the building automation systems must then be tested against this specification based on the BACnet standard.

The following standards and test specifications are the basis for carrying out such tests: DIN EN ISO 16484-5, ANSI / ASHRAE 135-2004, ANSI / ASHRAE 135.1 + Addendum-A (future standard DIN EN ISO 16484-6 Testing Conformance to BACnet), ISO 9646 Part 1-7 (Test Procedures and Conformance Checking Procedures, Not Considering Performance or Error Behavior, serves as the basis for 135.1 and the series of tests to be performed.), ANSI / IEEE 829 (document format for displaying test results), BTL-specified tests (In This document describes additional tests specified by the BMA.), user-specific guidelines and network specifications.

The American Association BMA (BACnet Manufacturers Association, www.bacnetassociation.org) operates a test laboratory called BTL (BACnet Testing Laboratories). The BMA has been preparing test plans for some years, which allow tests to be carried out on the basis of the standard (ANSI / ASHRAE 135-2004 / BACnet or DIN EN ISO 16484-5) and has already carried out manual BACnet test series.

Tests on the interoperability of various devices from different manufacturers are organized in the USA and in Europe in the form of so-called "plugfests". In doing so, functions of the standard newly implemented by manufacturers in devices are checked against existing implementations. The investigation of interoperability takes place above all taking into account the user-specific requirements with regard to the quantity structure of the supported object types, telegrams etc. as well as special BACnet functionality in Europe or in the USA. So far, tests for conformity to the BACnet standard, interoperability tests and performance studies are only possible in a manual and functionally limited form.

Due to the high demands placed on functionality in a large property, the effort required to perform manual tests is extremely time-consuming and costly due to the large number of possible BACnet functionalities. Thus, competition via BACnet between several GA products at a high functional level is hardly possible because the required functionalities can not be completely tested with reasonable effort. Therefore, the equipment of only one supplier must be used, from which the homogeneity and interoperability of its devices in the network is expected. Automated tests (i.e., software-based tools) for conformance, interoperability, and performance relative to BACnet are not possible in the prior art.

Therefore, there is an urgent need to reduce the testing effort and streamline the execution of BACnet test series.

This is enabled by any of the methods according to claims 1, 3, 9 and 16. The methods can be combined with each other for a further increase in efficiency. Advantageous embodiments can be found in the dependent claims.

The inventive methods enable each operator to rationally test and approve BACnet devices for their respective properties and project-specific requirements. The at least partial automation of the tests makes it possible to check devices from different manufacturers within a reasonable time frame at manageable costs. Only then will an efficient method for the extensive certification of BACnet components be created. The rating of

Building automation equipment based on the criteria of BACnet conformity and their interoperability in combination enables the operator of large properties to specifically select components from different manufacturers. The documentation of the test results also offers the possibility of dialogue with manufacturers to specifically expand and improve their products with regard to the specific requirements of the user. The timely feedback of the test results to the manufacturer thus completes a control loop which ultimately leads to achieving the homogeneity of the products of different manufacturers aimed at with the BACnet standard.

Furthermore, the flexibility and parameterability of the test method according to the invention allows the application to projects of any size and different functional requirement and thus a test of the building automation devices of any number of manufacturers, taking into account the respective user-specific requirements.

The automated tool makes it possible for all properties with economically justifiable effort to verify the property-specific requirements taking account of the standard across all manufacturers. In doing so, user-specific requirements, which can be freely defined by the respective properties (especially in large properties such as airports and building complexes), can be taken into account.

For the first time, this method allows property operators of a network with BACnet building automation systems from different manufacturers in a network network to use the BACnet communication protocol with the required depth in a manufacturer-neutral manner.

In the following the invention will be described in more detail by means of an embodiment and the attached figures. Hereby show:

- Figure 1 shows schematically a flowchart for the automated test for conformity

FIG. 2 shows the message of a conformity checking program in the event of cancellation;

FIG. 3 shows a possible user interface during a consistency test as part of the conformity check;

FIG. 4 shows a window of a test software during the execution of the test plan in the context of the conformity check; FIG. 5 shows the schematic sequence of a server test within the scope of the conformity check;

FIG. 6 shows the schematic sequence of a client test in the context of the conformity check;

FIG. 7 schematizes the course of an interoperability test; and

- Figure 8 illustrates a performance test.

The method according to the invention consists in an evaluation of the properties of building automation devices according to the criteria of BACnet conformity and their interoperability in combination within large properties. The components of this procedure are a conformity test, which is carried out separately for each GA unit, and an interoperability test, which is carried out with two GA units in combination with each other, which have passed the conformity test. The test results are documented and reported back to the manufacturer of the GA units. Preferably, a performance test is additionally performed in a network environment.

The individual aspects of the process, which in turn are procedures of independent importance, will now be described separately.

Method for automated testing for conformity to the BACnet standard taking into account user-specific requirements

Figure 1 shows schematically a flowchart for the automated test on

Conformity, based on a test software. First, the conformity to the BACnet standard has to be checked. To keep the effort as low as possible, only the actually required functionalities are checked. For this, the operator of a property to be equipped with building automation equipment is provided with a test specifications ("GA specifications") containing the user-specific requirements, for example also the settings to be used according to BACnet standard.

The device manufacturer or manufacturers provide so-called test units under consideration of the GA specifications. As a test unit, a single or a composite of several BACnet devices is understood, which was prepared according to the requirements (programming / parameterization). Together with the test unit, the manufacturer provides a documentation of the operating steps as well as a device description in the standardized format "EPICS" (Electronic Protocol Implementation Conformance Statement document (EPICS, see ISO 9646)), which announces the degree of its BACnet implementation The EPICS fulfills the function of the PICS and PIXIT documents in electronic form in accordance with the ISO 9646 standard.

In the case of the automated conformity test according to the invention, the user-specific requirements, as reproduced in the GA specifications, are first of all entered in a test database as a default. The EPICS file provided by the manufacturer is read in as a template by a test software. Preferably, a syntactic examination is carried out. In the case of syntactical errors, the file is rejected and the import is terminated, otherwise the data is stored in the test database. By way of example, FIG. 2 shows the message of the program in the event of an abort. In the next test step, the manufacturer's specifications in the EPICS document are automatically compared with the user-specific requirements specified in the test database. Unsupported functions, objects and properties including their BACnet formats of the test unit are evaluated as negative test results and stored automatically in the database.

In the following automated test step, a so-called EPICS consistency test according to the ANSI / ASHRAE 135.1 standard is carried out. The test steps for the EPICS Consistency Check are defined in 135.1 and stipulate, among other things, that a check must be carried out in such a way that all the functionalities described in EPICS are actually present in the test unit. This is done by reading out the functionalities from the test unit and then performing a software-related comparison on set and actual state of the functions. For example, if a manufacturer specifies in his EPICS to support the BIBB DM-DOB-A and if this is not found in the test unit (criterion: reading the property Protocol Services Supported from the device object), then there is an inconsistency between EPICS Document and device implementation (or device configuration). These inconsistencies are documented in the checklist and as negative test results scored. FIG. 3 shows the possible user interface during the consistency test.

In the next test step, based on the BACnet functions implemented in the test unit, automatic generation of a test plan in the test software will result from the BACnet services and BACnet objects to be tested, including the user-specific optional properties and the application-specific required read / write access , For this purpose, the individual test steps described in the standards (for example ISO 9646) are used. With the aid of the test software, the positive and negative test results of the individual test steps are automatically checked and documented. By way of example, FIG. 4 shows the window of the test software during the execution of the test plan. The test database contains script files for carrying out the respective tests. The test scripts are preferably developed using the freely accessible scripting language PYTHON.

The test results are stored in a database and can be exported to Office applications for evaluation. Individual results can be overwritten manually, but the original result is retained. This serves to evaluate a fact that is not clear in the standard, which must be examined further, as positive at first. In individual cases, however, a positive test result can also be negatively evaluated manually if e.g. after proper execution of the single test the device is restarted or problems with the device have been noticed, although the actual test result was positive.

The results of all exams are in one coherent

Report compiled and documented deviations from the underlying standards and user-specific requirements.

Specifically, a manufacturer submits a test unit, for example, which, as specified in EPICS, supports the BACnet function DM-BR-B (Device Management Backup / Restore as provider). This means that the test unit must have at least one BACnet file object. The first test step (testing for user-specific requirements) would thus be fulfilled for this individual function (requirement by the operator of the property, provision in the test unit based on the manufacturer's information).

If it is determined in the second test step that, contrary to the specification in EPICS, no BACnet file object exists, then the test is terminated negatively at this time. Also, the test step one is thus to be considered negative.

Otherwise, in a third step, all tests specified in standard 135.1 or the accompanying standards will be carried out automatically. The device behavior for positive as well as negative tests is clearly defined.

An individual function (examination of the individual BACnet BIBBs) can only be considered as passed if all individual test steps have been successfully completed on the basis of 135.1. In standard 135.1, for each BIBB to be tested (BACnet Interoperability Building Block), various individual steps are stored, all of which must be performed in order to check the conformity for this BIBB.

The test steps include both positive tests, i. the test software sends a valid telegram with correct data content and expects the test object to receive a valid result, which can either be automated via the BACnet protocol, e.g. by a list of valid return values or by e.g. Reading a value e.g. on an operating display (current room temperature, etc.) is verified by the operator of the test software.

In the negative tests, although correct data telegrams are sent syntactically to the examinee with an invalid data content. Subsequently, it is verified whether the device under test behaves correctly according to the specifications of the standard, e.g. returns a correct error code.

Ideally, a test unit from a manufacturer fully complies with all the requirements of the GA specifications, standards 135.1 and 16484-5 and the accompanying extensions.

As a case study, the test steps for testing ReadPropertyServiceExecutionTests in the area of application execution should be considered

Tests serve. These test steps verify compliance with the Area of the BIBB DS-RP-B and serve here as an example for all BACnet BIBBs.

Positive Tests ReadPropertyExecution

Two individual tests are specified in this area:

Reading the size of an array

Here it is checked if the size of an array is processed correctly.

Reading a single element of an array

Here it is checked whether the readout of a single element of the array is processed correctly.

Negative tests ReadPropertyExecution

Four individual tests are specified in this area:

Reading non-array properties with an array index

Here, a check is made as to whether the device under test supplies a valid error code when attempting to read from an element (property) specifying an array index, where the referenced property is not an array.

Reading Array Properties with Array Index that is Out of Range

Here, a check is made as to whether the device under test supplies a valid error code when attempting to read from an array specifying an array index outside the range of the array.

Reading to Unknown Object

Here it is checked whether the examinee supplies a valid error code when trying to read an object not present in the test object.

Reading to Unknown Property

Here, a check is made as to whether the test object provides a valid error code when trying to read out a property (property) that does not exist in the object. The test cases specified in 135.1 as well as all test procedures and result documentation are carried out as far as possible automatically within the test software. In addition, manual tests are also performed during the tests. For example, the manual tests refer to the evaluation of read (read by the operator) values on a display, mainly in the area of the BACnet client functionality. These are also marked and specified as such in 135.1.

Based on the error codes or expected results specified in 135.1, the evaluation is made as to whether a test object has successfully passed the correspondingly specified individual test. A single step can only be considered successful if the result set is in the range of the 135.1 specified results, with manual readings possibly in a specified value range.

Advantageously, the method is carried out by means of a test software program for which a standard IBM compatible PC (at least 2 GHz, 512 MB

RAM) with a Microsoft Windows 2000 or XP Professional operating system. Depending on the data link layer, corresponding hardware or software drivers, e.g. RS-232, network card or TCP / IP required.

The test software works as a BACnet client and as a BACnet server and can test both BACnet server devices and client devices. The

Script processing is configurable based on ASCII files (Python scripts), a separate script management of the test procedures is possible. Of the

Test generator works on the basis of predefined scenarios with a

"Request / Expected / Entered" logic, e.g. Request: Who-Is, Expected: I-Am, Entered:?.

If the test software works as a BACnet client, it will largely automatically query the BACnet server to be tested in accordance with the test specifications of standard 135.1 and the user-specific requirements and record the results. For certain interventions (eg switching operations on the device), the execution is stopped and the message box is used to wait for the test steps to be processed manually. In this case it is possible to enter a free comment text by the operator. The schematic sequence of such a server test is shown in FIG. 5. If the test environment works as a BACnet server that provides objects for the BACnet client to be tested, checking for correct client behavior requires manual operator input (eg "Value: 27.2 read") possible.

A new test session optionally starts with the reading of an EPICS

File (device properties) or with the selection of a device template ("Wish List", ie user-specific requirements of expected functions, regardless of whether they actually exist in the device).

The writability of the properties is specified by the BACnet Conformance Code per property: R = Required, O = Optional, W = Writable.

The EPICS only contains the properties available in the device, but a customer / operator of a property may want a device that must support the optional properties in order to be considered successfully tested (implementation of customer-specific requirements). For this, the database provides a reference to individual tests in protocol reviews 135-2001 and 135-2004. These are stored in the database with an indication of the wheel and file name on an external directory / file. Future BACnet revisions can also be implemented in this way.

By assigning these tests to the objects of the device under test, the individual tests of the test plan are created as a list. If a test contains one or more dependencies, the dependent tests are also assigned automatically. If the selected tests are not to be applied to certain objects / properties, these objects or properties can be deactivated in the operating software (GUI).

Operator input starts generation of the script-based test plan, i. The GUI generates the test plan to be executed later using the device information and the associated tests as a Python file. This method has been specified to logically separate the data to be tested (objects / properties) from the actual test functions (e.g., WritePropertyMultipleTest).

In addition, the total amount of tests contained in the test plan is determined and assigned to the value 100% for later percentage display. On operator input ("Start-Button"), the interpreter is called with

Specification of the complete test plan to Python (path + file). The to

Communication with the operating software to be used TCP port + IP is read from the configuration data of the interface and appended as another command line argument to the call.

Example of a call: python "C: \ Tests \ Testgeratxyz.py" 127.0.0.1: 5050

The operating software now expects messages from the Python interpreter about the

TCP interface. After receiving the end message, the processing of the

Test plan completed. The result values of the individual tests are also transmitted with a separate message and are used in the

Database stored test plan assigned as result set.

In order to be able to execute a test plan several times with the same device, the results of the actual test plan are kept separated with the time stamp of the test execution.

For client testing, it is necessary for the test environment to be a BACnet

Server object provides. This is also defined in the operating software. For this purpose objects, properties and values of the simulation server are defined. These properties can be saved as templates for later recall. After starting the test plan, therefore, the server device is first generated based on the defined configuration. The schematic sequence of such a client test is shown in FIG. 6.

Although in principle the scripts could also be controlled via direct calls from the operating software, the use of the Python interface is an essential part of the reproducibility and serves as a proof of performed tests. An accredited test laboratory is required to provide evidence of tests performed. Since the operating software creates these tests in readable form (py file), the evidence is given.

Tasks of the interfaces:

1 = call Python with specification of the test plan file as well as the TCP port for communication of the messages to the interface The GUI starts the Python interpreter, as arguments the name and wheel of the test plan file as well as the TCP port to be used for the messages to the GUI are passed.

2 = Stack API.dll BACnet communication

Providing the encoding and decoding functions of the BACnet

Telegrams as well as implementation of the TSM (Transaction State Machine) with the possibility to transfer all telegrams instead of answering these directly to the calling program parts (Python) for processing.

3 = message interface to the surface

This interface is used to implement a progress bar in the GUI and to synchronize between GUI and Python. Only messages from Python are sent to the GUI.

The following messages are transmitted:

percentage step completed (used to display a percentage of the surface area of the total

Individual tests) (see percentage display)

-Startmessage: is sent when a new call of a test takes place

- return values of the test (see error codes)

- Endemessage (All tests completed)

- ping message to the interface (vital signs), in the absence of this ping after timeout, the GUI detects that Python is no longer running and aborts with operator's note.

- Operator input (read result): The operator was prompted to read a value during a test step within the execution of a test.

The value is stored in the DB together with the number of the test step.

- Operator input remark text: The operator has entered a free comment text in a MAKE statement. These inputs are stored in the DB and assigned to the test with indication of the test and test step.

Aborted by the GUI (operator abort), in which case the TCP socket is closed, Python then detects that the socket can no longer be written to and terminates the further execution of the script and the interpreter.

4 = database interface

The DataDank contains the references to the external py files of the tests as well as the test projects, device information, etc. The required data model results from the previous descriptions of the test procedure of this specification and is specified as a DB model.

With the help of standard office tools the test plans, tests, test steps can be referenced later (ODBC / database) and the results in the form of printouts, processing in tables, etc. further processed.

Four positive possible return values are specified for the evaluation: ok, error (function not compliant), not tested (test step was deactivated when the test plan was generated), not supported (test step contains functions that are not supported by the BACnet stack, eg VT- Tests), as well as negative return values, eg: Timeout when accessing the device (in this case, the interface may offer to cancel the further execution of the test plan) and Python not installed.

In the GUI, after the start of the test plan, a percentage indication with a cancel option is displayed. With each start message, the percentage is increased by the percentage step and the percentage is updated.

A test consists of a script coded in Python, which implements the individual test, for example, using the chapters in 135.1. Proprietary objects and proprietary properties require (if possible through the stack) proprietary scripts, ie they may need to be specially programmed. These tests should be added to the database and also deleted again (reference to py file). The conformity check is therefore largely automated and limited to the functionality required by the operator / customer.

Method for testing according to the BACnet standard for interoperability

After successful conformity testing, the devices or test units can be tested for interoperability. Both

Interoperability checks will determine the correct interaction based on the

BACnet standards and the user-specific guidelines of different components from different manufacturers.

For the evaluation of interoperability, test cases have been developed that can be used to assess whether and to what extent several BACnet devices work together correctly. Based on the test cases, all devices or

Test units in the network network checked for correct interaction. The

Test steps are monitored and evaluated using test tools (network analyzer, etc.). The log files recorded during the execution of the tests using the test tools are archived by the test personnel and, if necessary, documented. These log files allow a later assessment of possible problem cases and serve as proof of the

Exam.

The interoperability tests will only use test units already tested for conformity and provided with a positive test result. Since correct conformity of data telegrams and the behavior in the event of a fault have already been checked in the conformance tests, the focus of the interoperability test in the investigation is on the timing behavior of the devices and the matching application implementations of the individual devices in the practical test. Only compatible BACnet functions that are supported by both devices can be tested (common intersection). A distinction is made in the BACnet standard when specifying the BIBBS in A and B functions: A = requestor (client), B = provider (server).

Do both devices support matching BIBBs, e.g. the one device

DS-RP-A and the other device DS-RP-B, this statement says nothing about the supported quantity structure, ie the common intersection of both Devices off. The established interoperability is the data throughput supported by both devices. For example, a PC-based client could overload an automation station (DDC device, direct digital control) under extreme circumstances with requests in such a way that the communication due to telegram repetitions can be extremely slow up to the complete loss of communication.

The interoperability tests therefore check and document the reconciliation between different devices in order to achieve optimal interaction.

Furthermore, the BACnet standard only describes the communication in the ISO / OSI model layers 1 to 7, but not the implementation of the respective BACnet application. Here, e.g. a check for a different processing of the values in devices from different manufacturers, e.g. the processing of decimal places, etc.

To prepare for the interoperability check, all devices to be examined must first be integrated into a common network network, i. There is a physical and communicative connection from every device to every other device. In each case, the combination of the application implementations of two devices in this network is examined.

If a device provides a B function (e.g., ReadProperty-B), it must

Device that is being tested for Interoperability, have the appropriate ReadProperty-A client function. If there is no match in this interoperability area (IOB), this area can not be considered for interoperability. The full list of features supported by both devices will check for interoperability.

Each single function supported by both devices is a single interoperability testing step. At the application level, the provided value or the provided BACnet function (eg data backup) is read from the provider and documented. The requestor reads this value or executes the provided function at the provider. If the value read out corresponds to the previously determined value Result or the function was successfully completed by both sides, this single step is considered to have been successfully fulfilled.

If there is no match of both devices in the overall list for a certain area or if the check of the respective single step was not completed successfully, then this single step should be considered as unsuccessful.

If the devices are devices of different manufacturers, then in a following step it can be checked and defined which adjustments have to be made in the implementation in order to achieve an interoperability. As an example, one of the two manufacturers could increase timing or resource parameters to accommodate the characteristics of the other communication partner. The interoperability tests provide a basis for decision making for possible extensions of the device applications.

FIG. 7 schematically shows the course of the interoperability tests. First, the areas in which the devices fit together, ie BIBBs, network types, etc., must be determined, this is determined using the PICS or EPICS. From this a test plan is generated. For each test step, an evaluation is carried out successfully and not successfully and documented.

The tests to be performed start with simple functions such as reading or writing values (data sharing) to complex functions such as

Exchange of schedule, alarm, and trend data as well as special

Functions such as RemoteDeviceManagement (for example, restarting a device) or backup / restore.

Interoperability tests can not be automated, but are done manually using a connected network analyzer.

In the following, a BACnet control system or the client part of such and, as a server device, the server part of a BACnet automation station will be referred to as client device. Since the interoperability tests depend very much on the supported functions of the respective devices, the following questions should be answered in each individual test session: Device Binding Can the client device find its communication partner using the Who-Is and Who-Has functions and determine its properties? Attention is paid to special features such as static address binding in the run-up to the respective test session and the devices are configured to the respective test conditions by the cooperation of the manufacturers. The test is then successfully passed, if all the properties of all

Interoperability ranges (IOB) of both devices (common intersection) could be successfully read out. Data sharing (configuration and operation)

Can the client device complete all properties of all objects of the communication partner (including the optionally required

Formats) read successfully? Client devices may e.g. based on

Protocol_Revision detect which BACnet version of the

Communication partner is supported. The test is successfully passed if the client device is able to read the properties of all objects of its communication partner completely successfully and, if implemented, can set its communication according to the BACnet protocol revision.

Can the client device meet the required properties of

Influence (write) communication partners? For this test, all writable properties of the communication partner must be able to be modified. The test is passed successfully if all writable

Properties of the communication partner could be modified.

Data type tests: Can both communication partners support the same BACnet data types completely (optional formats)?

Choice Tests: Does the server device support the possible choices of one

Properties? Those properties that are defined as choices are compiled as a list and tried by the client with all defined choices describe. The test is considered passed if all options of all tested properties have been successfully written and accepted by the server device. (Note: this test is partly already part of the conformity test).

Alarms / COV

Alarm / COV Configuration: Can the client device affect alarms and COV in the server device and log in for alarms and COV?

Alarm / COV Reception: Can the client device receive alarms and COV from the server device? Is there a receipt acknowledgment for confirmed services?

Alarm list: Can the client device request an alarm summary (alarm summary, event information) from the server device?

Alarm acknowledgment: Can the client device acknowledge alarms (AcknowledgeAlarm)?

Schedule tests

Reading out the schedule configuration: Can the client device read out the schedules of a schedule object of a server device?

Schedule configuration: Can the client device influence Schedule objects?

Schedule Types: Both types of schedule are used by both devices

(Exception and Weekly Schedule) supported?

Schedule data types: Are both data types of the Schedule object (at least Analog, Binary, MultiState) supported by both devices?

Trendloq Tests Read Tests: Can the Clientger read a Trendlog object from the server?

Write tests: Can the client device start and stop the trend recording? Trend alerting: Can the server device inform the client device via alarm if the trend log memory threatens to overflow (EventNotification)?

Automatic Trend Reading: Does the client device read the Trendlog object due to an alarm message?

Device management tests

DCC Tests: Can the client device lock and unlock the network communication server device

(DeviceCommunicationControl)? Are the optional passwords supported?

Restart tests: Can the client device restart the server device (ReinitializeDevice)? Are both types of restart supported (coldstart / warmstart)? Are the optional passwords supported?

Time Synchronization Tests: Can the client device send TimeSynchronization and UTCTimeSynchronization messages? Can the server device receive and enforce these messages?

Backup / Restore Tests: Can the client device back up the server device? Can a data backup be restored to the server device using Restore? Does the server device behave the same way as at the time of the backup?

Network and routing tests

To perform these tests, one BACnet router is another

Network required.

RemoteNetwork tests: Can the client device find a server device over a remote network?

Service device tests: Can the client device communicate with a server device over a remote network?

Halfrouter Tests: Can the client device initiate and terminate a PTP connection through a halfrouter (EstablishConnectionToNetwork and DisconnectConnectionToNetwork)?

BACnet / IP tests Is BBMD supported by the client or server device? Is an FD registration processed? Is the FD register interval met?

BACnet / IP GLT functions

Can a workstation process (read and delete) the FDT (Foreign Device Table)? Can a workstation process (read and delete) the BDT (Broadcast Distribution Table)?

Method for testing according to the BACnet standard on performance

After successful testing for interoperability, the performance is checked.

Here, the offered BACnet components are tested and documented both in the test laboratory and in the user-specific IT network in terms of data throughput, response times and network bandwidth measured between their subsequent installation location on the network route used to the specified communication partners. The aim of the performance investigations is to examine and document network properties of the IT infrastructure in relation to the reaction times of the devices as well as to check for possible influence on the communication of a BACnet network. Test cases are developed that allow conclusions to be drawn regarding the expected performance of the devices in terms of network bandwidth, response times and overall throughput.

Each individual device is first subjected to an examination of reaction times without consideration of the entire network network. In a further test step, the network reaction time between the planned installation location of the device and the location of the communication partners is determined. In the subsequent test step, the test units are connected at the network site at the installation site via the network path used to the specified communication partners and the performance is measured. All test steps are documented and archived for later use and assessment.

For the performance studies suitable tools for measuring the time behavior are needed. For this serve on the one hand commercial

Tools such as network monitors and analyzers to the other additional BACnet-specific tools, such as a "BACnet-ping" software program The "BACnet-ping" program works in a similar way to the "ping" tool used in the IT environment, except that it specifically addresses the protocol-specific properties required for BACnet (UDP protocol, used port numbers, etc.) It allows the time measurement of a BACnet telegram with a defined data content over a defined section of the BACnet network.

All definitions to be determined for the performance check are given based on the expected data traffic between two stations (minimum requirement for network traffic) plus a safety impact in an entire checklist. Furthermore, the checklist contains the specification of a desired minimum requirement for the maximum permissible duration of each individual step. In different test steps, different amounts as well as different access points and sections of the network are defined for subsequent testing.

During the test, the defined in the respective test step amount of BACnet telegrams is sent with a defined data content from a defined point of the network to another defined point of the network over a defined section of the network. The time required to transport this amount of data is logged at both access points to the network and documented in the checklist. Figure 8 illustrates the performance test.

Does the temporal behavior correspond to the desired temporal

Minimum requirements, this test step is considered successfully. Used to transfer a defined amount of data over a defined distance of the

Network requires more time than expected, this test step is considered unsuccessful.

As part of the performance reviews, different "message loads" (different packet sizes and packet volumes) of the network are used to assess potential impact on the performance and overall performance of the system Latencies in the network, selection other access points, etc. are defined. The

Performance examinations thus serve as a basis for deciding on the selection and specification of the BACnet devices to be used and for determining the network requirements.

Performance considerations in the network environment are basically divided into three areas: Performance analysis of the server devices, the client device and the network.

In the case of the server devices, the goal is to determine the maximum time required for the communication of individual, comparable information between individual information points of a server device over the entire signal path ("worst-case view"), eg analog actual value, digital setpoint value, multi-level value, etc. The term "entire signal path" is understood to mean the entire path from physical input or output to the end point of the communication (eg control station). Timing must therefore begin by triggering the desired action (e.g., actuating a switch or altering the measurement) and measuring the time to time the message arrives at the desired endpoint (e.g., at the building control center GLT).

The maximum time required is therefore determined as follows:

- The server device to be examined is configured with the maximum number of objects to be managed specified by the manufacturer. This is important for considering the maximum expected latency.

- For the preparation of this test series, data points are selected for the determination from the overall configuration, of each data point type at least three pieces are consulted, in each case the first data point per data point type related to the configuration of the device, the last and any selected point within the data point range. - The selection is made on the basis of a data point list that must match the configured objects in the device.

- For the execution of the test, the device to be tested in the

Network introduced (installation for testing purposes). The structure of the total distance must take place over the longest possible route or the slowest communication path of a network (here are possibly dial-up connections in a network (additional connection setup phase), ping times in one

Network, etc. to consider.

- The device to be examined is placed at the defined point in the network and installed communicatively. This test execution slot has the greatest distance or slowest connection in the network relative to the desired endpoint (e.g., GLT).

In a step to be triggered manually, the data points are triggered individually and the time taken for the end point is measured using two persons with a permanent speech connection (for example via telephone or radio, etc.). One person triggers the data point, the second person triggers a stopwatch and stops it on arrival of the expected value change at the endpoint. In order to achieve as little delay time as possible, three times are counted down via the speech connection before triggering (synchronization of "transmitter" and "receiver"). To ensure that the measurement result is as accurate as possible, the same test is performed three times per data point and averaged. Latencies in the voice connection, etc. are ignored in this model.

- The expected accuracy of this test series is lower Seconds.

To determine comparison values (without any delays or

Influences from the network), this test series is also carried out in a small network structure without additional components. These

The examination is carried out with one reference device each taking into account the maximum possible configuration.

For client devices the aim of the test is the simulation of a high utilization of the control technology (BACnet client) by high data transfer on the part of the BACnet server.

For this purpose, BACnet simulation programs generate a permanent and predictable "telegram flood." The aim of the measurement is to determine the reaction times of the control technology by connecting a defined number (eg 10-15 pieces) of BACnet servers in a small network be influenced by means of control parameters of the simulation server.

On the part of the control technology, the values are visually compared with the currently simulated values of the servers. Expected is a "run-on" of the values in the display of the client, which increases with increasing network traffic and decreases with decreasing network traffic.When there is too much data traffic (no more amount to be processed by the device), telegrams can be expected no longer be answered and therefore must be repeated.A failure of a device or other misconduct of the device are also conceivable effects.The visual control (stopwatch), a measurement accuracy in the lower second range possible.The simulation devices must for practical implementation of the test series in the control This requires the support of the BMS supplier.

For a performance analysis of the network, the average load of the network (utilization) as well as the available bandwidth should be considered. Since ICMP is only very suitable as a network protocol for measuring a maximum time period for the transmission of a packet because routers can be configured so that certain protocols are transported prioritized, another form of verification is necessary. The check is therefore based on the UDP protocol used in BACnet / IP. A test program ("UDP-ping") that can run on a PC sends a defined number of packets of the maximum possible message length of 1,476 bytes to a remote PC, each of which sends back a defined response telegram The averaged time for all packets and telegrams divided by two gives the average delay time in a network.

So that it can be tested under extreme conditions, this is done

Test series under both "normal" and extreme conditions at high network load

Data backup over the network or on the occasion of a similar one

Operating situation with high network utilization done.

The accuracy of these measurements is in the millisecond range because the tests are automated based on the PC timebase. These series of tests are carried out several times in succession, the average latency over all tests being statistically averaged.

Claims

claims

1. Method for the evaluation of BACnet

Building automation devices according to the BACnet standard within real estate, in which at least partially automated by test software:

the BACnet building automation equipment is individually tested for conformity with the BACnet standard;

BACnet building automation devices that have passed the test for conformity and that have common BACnet functionality are brought together and tested for interoperability;

the test results are documented.

2. The method of claim 1, further comprising performing a performance test.

3. A method for automated testing of BACnet building automation devices for conformity to the BACnet standard taking into account user-specific requirements, in which - user-specific requirements are recorded in a test database, a device description (EPICS) belonging to the BACnet building automation device to be tested is imported in electronic standardized format , - the information in the device description (EPICS) with those in the

Test database against defined requirements, the properties of the BACnet

The building automation system is read out and compared with the information in the device description (EPICS), upon successful comparison of the properties of the building automation device to be tested with the information in the device description, a test plan is generated based on the read-in device description (EPICS), which defines individual test steps, - the test steps are executed one after the other; and the test results for documentation are automatically stored in the test database.

4. The method of claim 3, wherein prior to reading the properties of the building automation device to be tested, a consistency check of the device description (EPICS) of this building automation device is performed.

5. The method according to claim 3 or 4, wherein the test steps comprise the test steps specified in the standard ANSI / ASHRAE 135.1 or DIN EN ISO 16484-6, after which corresponding data telegrams are sent to the building automation device to be tested and the return values obtained from the building automation device to be tested be verified.

The method of claim 5, wherein return values are compared to a list of valid return values.

7. The method of claim 5 or 6, in which syntactically correct data telegrams are transmitted with invalid data content to the test unit to check whether the test unit returns a valid error code.

8. Software program for carrying out the method according to one of claims 1 and 3 to 7.

9. Method for testing building automation devices according to the BACnet standard for interoperability, in which the building automation devices to be checked are integrated into a network in a network, for each two test units of the group, one of which is a requestor (client) and the other provider (server), all BACnet functions supported by both building automation devices are tested one after the other; - for each test step a rating is successful or unsuccessful; and the test results are documented.

10. The method of claim 9, wherein the provided by the provider data value is read out and documented and read by the requestor.

11. The method of claim 9, wherein the provided by the provider BACnet function is read and documented and executed by the requestor at the provider.

12. The method according to any one of claims 9 to 11, wherein in an unsuccessful test step in a subsequent step, an adaptation in the

Implementation of one of the two audited building automation devices is proposed to achieve a successful interoperability test result.

The method of any of claims 9 to 12, wherein the testing steps include checking complex functions such as the exchange of schedule, alarm, and trend data.

14. The method of claim 9, wherein the network also comprises a BACnet router.

15. The method according to any one of claims 9 to 14, which is carried out only after successful conformity testing according to one of claims 3 to 8.

16. Method for testing building automation devices in a network according to the BACnet standard on performance, in which For each test step, the set amount of BACnet data telegrams with a defined data content from a defined point of the network to another defined point of the network over a defined one is defined for successive test steps respectively different sets of BACnet data telegrams and different access points and sections of the network Section of the network, which is logged in each test step for transporting the amount of BACnet data telegrams between the defined points of the network time and documented in a checklist.

17. The method according to claim 16, wherein a provider to be checked is configured with a permissible maximum number of objects to be managed.

The method of any of claims 16 and 17, wherein a provider (s) to be tested is installed at a location in the network that has the greatest distance or slowest connection in the network relative to a desired endpoint.

19. The method of claim 16, wherein an overload of a requestor to be tested is simulated by a persistently high number of data telegrams.

20. The method of claim 16, wherein with the aid of a software program using the UDP protocol, a packet of a maximum possible telegram length sent from an access point and using the software program from a remote access point, a defined response telegram is returned, the time between sending a packet and the receipt of the response telegram is measured.

21. The method according to any one of claims 16 to 20, wherein a

Defaults list is generated, taking into account the expected

Data volume A maximum permissible time duration for the transport of the quantity of BACnet data telegrams between the defined points of the

Network pretends.