What is MADARA KATS?

MADARA is a focal point of my job talks, and KATS is one of the most exciting tools available in MADARA. In a nutshell, the KaRL Automated Testing Suite (KATS) is a portable deployment and testing system meant for automated sequencing or testing targeted at distributed, real-time and embedded systems.

What separates MADARA KATS from the rest of the pack is that the entire system is done in a decentralized way. This may not immediately seem important or interesting, but this opens up fine-grained control, support for fault tolerance, and responsiveness that can't be found anywhere else, especially in centralized solutions--e.g., solutions that use a centralized controller.

The features of KATS are itemized below:

• Fully decentralized system that targets large scale testing across multiple machines in a local area network
• Portable to most operating systems (Windows, Linux, Apple, etc.)
• Control over launched application
  
  1. Executable, command line, environment variables and many other application inputs
  2. Kill time and signal (on Windows, only terminate is available)
  3. Real-time class for elevating process priority
• Batch processing with parallel or sequential execution
• XML configurable
• Domain-specific modeling language available for modeling in GME
• Ability to instrument Android smartphones via both Monkeyrunner (MADARA MAML library) and ADB (MADARA MAAL library)
• 8-phase process lifecycle (See figure below for visual)
  
  1. Barrier (optional)--require that a group of processes come to a barrier before application launch
  2. Precondition (optional)--require that a condition is met before application launch (e.g., if another process succeeds or fails in one of its lifecycle phases)
  3. Temporal delay (optional)--operating system portable sleep time
  4. Post delay (optional)--set a global condition or perform logic that indicates you are past temporal delay phase
  5. Application launch--launch an application
  6. Post launch (optional)--set a global condition or perform logic that indicates your application has been launched.
  7. Post condition (optional)--set a global condition or perform logic based on the return value/exit code of your application.
  8. Exit
• Built-in network transports for RTI DDS and Open Splice DDS. Other transports can be added via expansions to 2 functions in the Transport.h file
• host agnostic--i.e., you can deploy whatever you want wherever you want due to the usage of anonymous publish/subscribe network transport layer.
• fault tolerant--i.e., you can deploy multiple failover entities in case of faulty hardware or whatever might cause a process to fail. Additionally, you can create tests that detect and respond to faults/failures.
• Nested tests and application launches
• Microsecond precision between process lifecycle phases that are not dictated by blocking communication of a centralized controller.
  

Now, the microsecond precision is important for DRE systems, especially in reproducing race conditions. With KATS, you can get the results of a postcondition in a failed application launch at the relevant precondition for another application launch within fractions of a second. With this open-source, freely available framework, you can perform black-box sequencing at scale.

Additionally, there are whitebox tools available to allow for distributed breakpoints within an application and powerful, thread-safe logging APIs in case those are needed. However, most people seem more interested in the blackbox testing tools.

If you have questions about the MADARA KATS system, feel free to contact me at jedmondson (at) gmail.com.