Community Blog

Hello, this is Todd Law again, vice-president of NTAF and senior product manager at Spirent Communications. Last time I wrote about the various reasons people choose to automate, but this time I wanted to ask an even more fundamental question, namely, what is automation?

This question may seem strange to ask since clearly most of us have some feeling and understanding for what automation is. But if you think about it for more than a few seconds, the answer to the question is not necessarily so clear.

Perhaps let me start by stating what automation is not.

Automation is not the writing scripts or programs. There is a common perception in the networking space where automation is often equated with using the API of a product. Most test tools do indeed provide both a GUI and an API, and both are valid means to using those tools. However, we should not confuse the means for automation with automation itself. It is, in fact, possible to automate using a GUI interface, and also possible to drive many test tools by firing off Tcl commands one by one. With that important distinction being made, I do of course accept that automation is frequently achieved by writing scripts that utilize test tools’ APIs.

So having separated the means by which automation can be achieved from the idea itself, let’s now attempt to state what automation is. If you look it up in a dictionary, you will find a somewhat self-referential definition that says “automation is the act or process of getting a machine to be more automatic”. If you look up “automatic”, you will find something like, “capable of operating without control or intervention”.

If we glue those two definitions together to come up with a single, unified definition (and taking some liberties to simplify things a little) we could then define automation to be “the process of getting a machine to be more capable of operating on its own”.

It is worth noting that the definition uses the word ‘more’ which is a relative term. I like this, because it reflects the reality of automation in that the goalposts keep moving. For example, in the network test space, traffic generators started off as fairly simple frame blasters (which were themselves an extension of even more basic bit blasters). Gradually over time, however, traffic generators have added the capabilities to emulate a plethora of protocols, which are, in a sense, a form of automation, in that packetized responses are sent automatically from the tester to the device under test in response to the stimuli of specific incoming packets. Such emulations hold up one side of a protocol conversation with a counterpart in the device under test. Emulations have been a natural evolution of test tools as the ways in which the Internet is used has exploded over the past decade. Emulations provide the ability to test higher-up layers in the 7-layer OSI model of networking, and the various services that depend on underlying infrastructure.

Supporting multiple emulations is now the norm, and the expectations for automation continue to rise. Routers, and the tools used to test them, now more resemble high-end computers than simple hardware designed primarily for the narrow-purpose of forwarding packets. Routers are now capable of doing so many things that the number of test cases usually reaches into the thousands. Vendors and customers need to apply ever more automation simply to validate the vast scope of functionality, scale and performance, not to mention a continuing cascade of new features with each software release.

So how can network testing be more automatic than it is now?

Well, automation could be applied to many more parts of the network test puzzle: reservation, scheduling, inventory, topology, execution, inventory, reporting and authoring. All of these items are the stated priorities for NTAF. Moreover, since these problems frequently involve the multiple test tools as well as the devices under test, there is a clear need for standards in each of these areas to achieve the efficiencies possible via automation. Creating such standards, and driving their implementation, is the primary objective of NTAF itself. These are the problems we are working on, or will be working on, in the NTAF technical committee.

Automation standards groups, in the testing industry and beyond, pop up all the time. In each case there is a group of founding members that have a mission to make life easier for users of their products or solutions. It was no different when we came together to form NTAF in 2010. But NTAF is radically different from other groups, and the evidence was laid out for all to see during our first “plugfest.” My name is Kris Raney and I am the director of software development at BreakingPoint Systems, a founding member of NTAF. Today we are going to look at how far NTAF has already come in developing and promoting interoperability standards for the network test automation industry.

Proving Interoperability Throughout the Process

There are four key aspects of the NTAF effort that I think will make it a huge win, and they show why BreakingPoint works closely with the group:

1. It moves the point of automation from a programming API to a network-based API.
2. It makes use of the existing XMPP protocol, with XML-based messages.
3. It requires the interfaces provided by tools to be self-describing.
4. It’s being developed using a “show me how it works” approach, not a theoretical debate.

This last point is a defining difference for NTAF and why we don’t simply meet about these things and debate implementations. Instead, NTAF takes a proactive approach and puts these things consistently to the test.

Just a few months ago several of the NTAF members (BreakingPoint, Fanfare, Ixia, Spirent, and Verizon) put our initial draft specifications to the test. During the beginning of the week, each company chose the implementation language that made sense for them, and there were several—including TCL, Java, python, and an implementation that would work for any Windows CLR language. Then test tools that were located in two separate cities in California, two separate cities in Texas, and a server in the cloud started to interact. All of them were communicating through a variety of corporate firewalls, VPNs, and network security.

By the end of the very first day, these tools were functionally talking together, and by the time the demo came on Friday, they were working quite well together. The demonstration clearly showed multiple different combinations of cross-vendor interaction. Performing this live implementation of the specification not only showed us we were on the right track, but, more important, it also identified areas that needed improvement—and that’s the great thing about having made real implementations. The NTAF 1.0 specification isn’t being made in a theoretical vacuum, and these things will be fixed.

Very soon we will officially publish the NTAF 1.0 specification. Iterations of this specification have already been put to the test, as I mentioned, to show how NTAF is moving the test automation industry forward. And with the introduction of 1.0 we will continue to put our work to the test and proactively work together to produce a specification that is strong in its vision and in its capabilities.

My name is Jitin Dhanani and I am the Business Development Manager at Ixia.  I also hold the office of NTAF Treasurer and represent Ixia on the NTAF Board of Directors.

 Ixia is a founding Member of NTAF and has played a leading role in forging the path towards developing the first version of an industry specification.  As an industry leader, Ixia has always been a strong proponent of automation and open specifications based interoperability. We were a founding member of TesLA Alliance (formed in 2008) with some of the same goals as NTAF and played a pivotal role in publishing the TesLA 1.0 specification which has been submitted to NTAF for review and consideration for both the upcoming 1.0 version spec and for future specifications.

 The value proposition of Automation is well known and accepted. We have seen an increased focus on automation and its business need from our customers in the last three years. Primary business drivers such as Time to Market, Quality and bottom line benefits increasingly drive the need for an end to end automation tool to orchestrate the multiple pieces in an increasingly complex lab environment. The major obstacle to truly multi-vendor automation has been a lack of industry defined and adopted specifications.

 As with any specifications effort, the issues at the heart of this challenge are strategic, technical and political in nature. Successful specifications development requires that marketplace competitors come together to jointly define specifications – and then adopt them.  This has been the genesis and focus of NTAF as an industry standards body. NTAF’s membership includes key players in the networking ecosystem.  Besides test vendors such as Ixia, NEMs such as Cisco, Brocade and Ericsson and service providers such as Verizon and BT also participate.  The membership also boasts infrastructure providers, IT services and system integrator companies.  Their active participation has fuelled the progress that NTAF has made on the specifications formation effort.  In less than 18 months NTAF is on the road to deliver the 1.0 specification which focuses on automatic discovery and communication between various components in a network lab. This will be a great first step on the way to easing multi-vendor automation solutions and environments. 

 Though proud of this achievement, we realize that any specification is only successful if it achieves widespread industry adoption.  Ixia, as one of the leaders on the Technical Committee, is an active participant in all interoperability plugfests and has demonstrated interoperability for both its traffic generator and test automation and authoring tool based on 1.0 draft specification.

 NTAF offers members the opportunity to shape industry specifications, gain mindshare and ultimately marketshare from an ecosystem of vendors which adheres to these specifications. I urge interested parties to contact me (or anyone from the NTAF membership) to learn more about the important work being done by NTAF on a specification to benefit the Test & Measurement industry.  

 Join us as we put automation in action. Make a difference. 

 Thanks

Jitin

jdhanani@ntaforum.org

My name is James Murray and I am the Head of Ericsson Test Environments for the Americas.   As a Founding Member of Network Test Automation Forum (NTAF), we strongly support the efforts of NTAF and understand the importance of NTAF’s vision to “develop and promote interoperability standards in the test community”.  As one of the few companies worldwide that can offer end-to-end solutions for all major mobile communication standards, Ericsson knows that interoperability remains a key component of delivering seamless solutions. When Keith Kidd of Verizon first approached me, we discussed the challenges we both face in our respective test environments. As a Network Element Provider and as a Service Provider we face many of the same complexities and pressures to deliver solutions and services to the market. Having seamless interoperability in the development and testing phases would move technology faster to market and improve our margins! So if interoperability is essential for networks and services in the field, should it not also be essential during the testing phases?

Sometimes we take for granted the importance of interoperability in our daily lives and how critical it is technology trends, past, present and future. Interoperability is so much a part of our daily lives we don’t even recognize it anymore yet it drives so much of our global economy today. I read a very interesting interview by Dr. Craig Barrett from Intel on the importance of global standards. The interview had many relevant points, but I like this one the best:  

 “When you have common interfaces, common protocols, then everyone can innovate and everyone can interoperate. Companies can build their businesses, consumers can expand their choices, the technology moves forward faster, and users get more benefit.”

—Craig Barrett, Intel

 So how can the NTAF help fulfill such a demand? Well, NTAF technical specifications will establish requirements for device discovery, network topology specification, test case definition, results formatting, and other technical obstacles that will ultimately ease the challenges associated with assembling a test lab consisting of equipment from multiple vendors.  To that end, NTAF’s Technical Committee has already made very significant progress and demonstrated interoperability through a series of virtual plugfests.

 So where do you stand on interoperability? Is it part of your daily life? Is it in your future?

 “Vision without action is a daydream. Action with without vision is a nightmare.”

    – Japanese Proverb

 You can read more on Dr. Barrett’s interview here, http://www.intel.com/standards/execqa/qa0904.htm 

 Another good read that demonstrates the importance of interoperability is Ericsson’s White Paper on the “Future Role of Telecom” http://www.ericsson.com/res/thecompany/docs/whitepapers/101117_future_role_of_telecom.pdf

 Sincerely,

 James Murray

James.Murray@ericsson.com

My name is Andrew Armstrong, and I am a senior product manager at Spirent Communications, and vice-President of NTAF. As a founding member, Spirent has been involved in the Network Test Automation Forum from day one, and is proud to bring industry leadership to this groundbreaking organization.

At the very outset, NTAF hit the ground running with strong representation from key industry players in the network test space, including leading service providers such as BT and Verizon, and leading network equipment manufacturers such as Cisco and Ericsson. Furthermore, NTAF’s membership roster is growing steadily, with the most recent addition being Brocade, which officially decided to join NTAF at the end of last month. I understand that several more players have reached out to NTAF expressing interest in joining, and we expect to relay more news to you through this blog in the months ahead.

Why join NTAF? A skeptical reader could be excused for taking a “wait and see” approach to NTAF. There is always lots of talk about new standards, but tangible results and business benefit are often hard to come by. . My response to the skeptics is threefold:
1. NTAF is producing tangible results which can be leveraged immediately. Just last month, I was fortunate to witness the results of NTAF’s first plugfest, and the results were impressive. After only a couple of short technical sessions, NTAF members were able to demonstrate interoperability between products from competing vendors, as well as interoperability with existing scripted test assets. These different systems were located in 5 labs throughout the world. How much would your organization benefit from this level of interoperability?
2. The current lack of Interoperability comes at the expense of innovation. This creates a drag on every Equipment Manufacturer, Service Providers, and test vendor. The inability of existing complementary or competitive products to work together means that considerable resources are invested in automation projects to “glue” the various software pieces together. Wouldn’t it be better if we could free up those resources for more innovation? Innovation that means better test equipment, high quality networking equipment with more feature content, and new higher quality services.
3. NTAF standards will not lead to commoditization. In fact, if anything, NTAF is unlocking a new wave of innovation for our industry. Other parts of the networking industry have benefited immensely from standardization efforts. Where would the Internet be as a whole without the IETF, for example? I would argue that standardization has seen all parties benefit significantly. The work that NTAF is leading will actually provide new opportunities for competitive differentiation, especially to those companies that lead the change by joining and participating.

Spirent believes strongly in this effort and has chosen to be a leading member of NTAF for all the reasons listed above. NTAF is organized as a board of directors and two committees – technical andmarketing – with Spirent being very active in all three parts. The board of directors consists of representatives from all founding companies, with four companies holding officer positions. Spirent has held the position of vice-president from NTAF’s inception. Spirent has also been very active in the technical committee in numerous ways – most notable of which is our demonstration of a pre-standard implementation to drive Spirent TestCenter – the industry’s leading network test platform for protocol performance testing. And just to make sure the world hears about the good work coming out of NTAF, Spirent is also heading up NTAF’s marketing committee, which drives multiple programs to promote NTAF’s progress.

October is a big month for NTAF. The first pre-standard prototypes were the main attraction at our first plugfest. This landmark automation event included multiple vendors, including Spirent, connecting their prototypes together in an effort to demonstrate pre-standard implementations. These efforts were summarized in a recent press release as well as a more detailed test report describing this event.

With all that being said, we shouldn’t forget the most important detail. A lot of very intelligent and dedicated people from across the industry are driving these standards. However, there are still ideas and implementations to be discovered. Don’t miss your chance to help drive these standards. I’m sure that you wouldn’t want to be forced to implement something down the line on which you had no say or input. Whether you are a vendor, manufacturer, or a service provider, you have likely spent years thinking about how things could be done better. Now is your chance.

So, what do you say? Care to join us on this one-of-a-kind journey?

Sincerely,

Andrew
Andrew.Armstrong@spirent.com