EnVisionary ICARE

Software Development Engineer in Test (SDET)

Microsoft in Ireland and across Europe is always on the lookout for Software Development Engineers in Test (or as we like to call them, SDETs). Last year, at Microsoft Ireland alone, we hired more than 20 SDETs. And the number is only set to increase as Microsoft looks to increase its engineering presence in Ireland.

As we search for great-fit individuals, we hear some fundamental misunderstandings about what SDETs at Microsoft truly do.

In my experience as a staffing consultant, when engineers see the word “Test” in a job description, they assume the role is in Quality Assurance (QA). I sat down with Ankur Modi, an Ireland-based SDET, to find out what myths need to be extinguished.

Ankur says: “The SDET role is unique and in my opinion is one of the most misunderstood jobs in the industry. People often think of SDETs as manual testers, sort of an automated robot if you will. It is however amongst the most rewarding roles a tech engineer can have.”

In his own words, Ankur sets the record straight:

Myth #1: The SDET role is a manual tester.

Truth #1: An SDET is an engineer.

In the three years I’ve been in my role at Microsoft, I’ve learned more than 10 new programming languages, several new data design concepts, database systems, smart interfaces and product design, all while shipping several key products. If I’m not an engineer, then who is?

 

Myth #2: Software testers do not have coding skills.

Truth #2: Software testers know how to code.

The SDET is in charge of ensuring everything works and the product exceeds users’ expectations. The level of quality and confidence only comes if you know the product, the components, and especially, the code within it. You‘ve got to code – and you’ve got to know how to do it right. Think of it this way: You can’t test something you don’t understand. It’s like building a car if you can’t drive.

Myth #3: The SDET does not have a voice.

Truth #3: The SDET has an important voice.  You are the customer advocate.

The SDET is “the guy who knows everything, sees everything, and has the final word on how things happen.” To be specific:

• I work with the Program Manager to decide how the product should be designed.
• I work with developers to define the functionality.
• I get to talk with people who use the product, including our partners, to assess its usability.

Myth #4: In the software development process, an SDET just rubber-stamps the product.

Truth #4: In the software development process, here’s the difference among the two roles:

• A quality assurance pro follows a methodology and tests software at the end of the development process. The role is not as technical as an engineer.

• A software tester focuses on all steps of the development lifecycle and remains accountable for producing high-quality software.

Myth #5: The SDET has a peripheral role.

Truth #5: The SDET is at the heart of the process.

Consider this equation: SDET = Project Management + Passion for Customer Experience + Engineering Skills. If you’re passionate about software development, you don’t get closer to the action than that.

 

The many approaches to test automation

1. Record & Playback

We are regularly called upon for advice on designing and deploying testing automation tools and frameworks. Choosing the right automation approach is vital for ensuring proper testing, minimizing development cost and delivering the lowest possible ongoing maintenance expenses. This dialogue usually begins with a simple but important client question: which automation approach works best? Based on our experience delivering hundreds of automation projects, we can confidently answer: it depends. Below is an overview of four different but industry-standard automation approaches to help the client narrow down their options and support a more informed decision.

This is the most basic automation testing approach, with limited flexibility and long term utility. Record & Playback functionality allows testers to interactively record user actions and replay them back any number of times, comparing actual results to those expected. These techniques use only hard-coded data; the utility is very dependent on the automation tool used. Record & Playback can be used by non-coders if customization is kept at a minimum level and the verification points are graphical. Usage Record & Playback is very useful for basic automation projects of short duration like automated data entry or exploratory testing that feature re-record from scratch, have short test flows and automated data entry. Pros Record & Playback is an ideal way to get into testing automation. This approach has a number of advantages including low development cost, relatively simple design and ease of deployment. Cons The basic nature of Record & Playback comes with a functional price. This technique has limited performance (e.g., minimal test flow coverage), robustness (e.g., can break on build) and flexibility. Ongoing maintenance costs can be high especially when manual intervention is required.

2. Data-Driven Frameworks

 Data-Driven techniques involve the creation of test scripts to run together with their related data sets within a framework. The test environment settings and controls are not hard-coded. A Data-Driven framework is ideal when the workflow is repetitive but the data input is large –meaning different combinations of data are required to successfully test the application. A skilled team is needed to implement this approach. Usage This automation framework effectively satisfies more advanced testing needs like testing single applications in multiple environments with big data sets, a limited number of checkpoints and stable test cases. Batch run tests are possible with extra development effort. Decent support is provided for distributed testing teams. Pros Data-Driven frameworks have a number of benefits including: very good usability and re-usability of test scripts, solid test flow coverage and reproducible test results. Cons There are watch-outs in using these methods. Automation performance is based largely on the automation skills of the person(s) undertaking the implementation. These techniques also require regular maintenance (increasing costs) and manual intervention upon failure.

3. Keyword-Driven Frameworks, also known as table-driven or action word testing

 A Keyword-Driven framework is an advanced type of automated testing. Its methodology divides the test development process into two distinct stages: a design & development stage, and an execution stage. A Keyword-Driven approach is ideal for many projects, environments and data sets. However, it also requires sophisticated coding skills plus an up front investment in development time and tools to maximize value. The design of the framework can play a major role in determining the return on investment. Usage This testing method is suitable for many automation testing projects across multiple applications, platforms and environments, featuring large data sets, short & straight test cases and a limited set of checkpoints. Keyword-Driven frameworks are also ideal for batch run tests. Pros Keyword-Driven frameworks are a very good approach to leveraging automation. They have many advantages including: very good script usability and re-usability as well as test flow coverage and the ability to reproduce test results. Moreover, test development does not require sophisticated programming skills. Cons Higher performance does come with a cost including a significant upfront investment for the design and implementation of the framework. Keyword limitations restrict the complexity of test cases. Employees will require knowledge of meta-languages. These techniques also require regular maintenance (increasing costs) and manual intervention upon failure.

 4. Hybrid Keyword/Data-Driven Framework

 Hybrid frameworks are the most sophisticated types of automated techniques available. For example, they feature script-less test creation, the ability to import/export data from various sources, a variety of checkpoints, and the power to integrate external objects. They can generally accommodate grammar and spelling as well as information input. However, the upfront investment, specialized skills prerequisite and ongoing maintenance costs can limit its long term value and applicability for simple projects. Usage Hybrid Keyword/Data-Driven Frameworks are designed to satisfy an organization’s most wide-ranging automation requirements covering multiple applications, platforms and environments. They are suitable for testing situations with large and changeable data sets and data transitioning cases. These frameworks typically provide strong support for distributed testing teams. Pros Hybrid Keyword/Data-Driven Frameworks are among the most comprehensive and flexible functional testing frameworks available. They feature high usability, re-usability and test flow coverage and offer built-in consistency and severity validation. Cons There are some challenges with this approach including the need for a significant upfront investment in development and skills for design and implementation. Ongoing maintenance costs could also be noteworthy, potentially limited ROI. Finally, these frameworks may also be overkill for many testing projects.

 The above analysis offers a general rule of thumb on which approach to use. What method you ultimately choose will depend on your budget, timing, skills and environment as well as your project’s testing goals. Fundamentally, test automation is a software development activity; your team will need programming expertise. 

For further information please contact: info@envisionary.com