Re-imagining beta testing in the ever-changing world of automation (Article at


Check the article at :

Today’s beta test solutions aren’t working to develop products that meet users’ needs. It’s time for a new vision.

If you ask most people why beta testing and related tools are important, they’ll name things such as shortening of beta cycles, reduced time investment, increased tester participation, improved feedback loops and visibility. However, all the reasons why beta testing is so critical can be narrowed down to two major issues, both of which are predominant in the beta testing phase of the SDLC:

  1. The intersection of humans and technology
  2. Usability and quality standards

Read complete article here:







Omnipresent Operating System (OS) : Re-imagining the Next Killer Experience for future of OS.


Reincarnation in the Hinduism is one of the fascinating philosophical concepts that says that a soul or spirit after meeting a biological death is capable of leading a new life in a new body. Being a Hindu, I also believe in the transfer of one’s soul after death into another body which produces a continuous cycle of birth, life, death and rebirth through one’s many lifetimes in Samsara.

Though this is an analogy from the mythologies and scriptures in Hinduism and other religious books, the concept of the same soul is passing through different bodies is somewhat can inspire the experience for the next-gen Operating System (OS) — an ecosystem where the OS is “omnipresent” as a soul and allows the user to move through different “systems” and “devices”(even we can extend this to IoT contexts!) .

A user story might record this user’s perspective something as —

“I should be able to use my application uninterruptedly and seamlessly across various devices/system as I move through different devices due to certain needs”.

It is like soul transfer — transferring consciousness from one body to another body.  In the movie The Matrix(1999), the Agents were transferring themselves to different person’s bodies. One interesting thing to note here is that being transferred to a new body they remember their goals, their memory etc.

(Fig: In the movie, The Matrix (1999), the agents can transfer themselves into other bodies at will.)

The idea that I termed as Omnipresent Operating System is to share application sessions (along with all the state/session data etc.) from one system to another system that allows to continue the application running on the later system in exactly in the same state and using the same session without the user to start the application from beginning or using a different application session in the later system to use all the benefits of available in the later system. This helps to achieve unifiedseamless & omnipresence-experience across different systems.


Following are five use-cases represented in images :

In each one, we can see a daily life usage where the experience for the user are extremely simplified through the proposed solution.

Historically the OS concept has progressed over time through many evolutions. Typically the OS is defined as a system of software which manages the hardware resources of the system to provide a base for its users’ programmatic computing needs. Throughout the history, multiple dimensions were addressed for operating systems such as performance, multitasking, usability, portability, mobility etc. We are witnessing a period of time when the transformation is happening to the evolution of OS at the highest rate possible. This is due to diversification of software technology, hardware, and evolution in new age eco-systems and new paradigms of digital devices. This is an age when we are witnessing the coming of IoT (i.e. Internet of Things) and cloud, where any device can be part of a bigger eco-system and be an extension of a cloud system.

So the question is what is the future of Operating system? Is it that cloud will be the ultimate operating system? At least by looking at the Chrome book, defining a thin-client based access to all the computational needs that stay in the cloud”. But still, the diversification of operating systems prompt us to pause and think, something is missing. The missing piece is convergence.

We have many derivatives of different types of OS. Android, iOS, Linux, Windows and OSX are to name the few of the variations, that continue the OS war. But users are limited by this. Imagining an application running in one OS . Can we use the same application in another one? No. Also imagine a situation, when you are reading an email and want to update it (the same email )on your PC…can you do it? No.

If we see the trends today towards the future, it’s all about micro-services and server-less architecture similar to torrent (peer-to-peer) or blockchain implementation, de-centralized and distributed eco-systems where the systems communicate to each other. In such scenarios, the next killer experience is to have the ability of the omnipresence across this distributed & de-centralized eco-systems.

Solution Implementation:

The solution can be implemented in two ways. To illustrate this assuming that there are two devices and the user initially starts using some application in the first one and then he moves to the second device and uses the same session and input data in the second device .

Approach 1. Both the devices (i.e. Device 1 & Device 2) will be running same Omni-present Operating System ( in real life scenarios it might be possible in some cases that they can be of different versions of the same OS ) which will have capabilities to transfer the application & user data including session/environment variables etc.. along with necessary components that will allow to render that in another device.

Approach 2. Both the devices may not run the same Omnipresent Operating System… they may be running 2 different OS (including 3rd party OS like Android and Windows etc.) .. in such cases both the device can have some Run-time Component installed to allow them to have the capabilities to transfer the application & user data including session/environment variables etc.. along with necessary components that will allow rendering that on the device on receipt of such data .

The following diagram illustrates the different fundamental components are to be placed in an architecture of the eco-system in order to make it work.

Description of the diagram :

100 and 200 are two devices connected over the same network via wifi/Lan/Bluetooth etc. (e.g. 150 & 250). 
110 is an application running in 100. 120 is the Runtime Component that helps to achieve the desired outcome proposed in the invention (might be part of the OS). 
100 is connected to internet via port 140. 
111 is the session data, 112 is user authentication and related data, 113 is environment related data (e.g. browser version, OS version, history, some system variable etc. ), 114 is any other metadata that might be associated with any of the components of the environment. 
115 is the application data (e.g. client session, user input data, cookies, local storage, application related variables etc. ). 
210 is an equivalent application running in 200.  
220 is the Runtime Component that helps to achieve the desired outcome proposed in the invention (might be part of the OS). 
200 is connected to the internet via port 240. 150 is a means of connection (e.g. wifi, LAN, Bluetooth, NFC, IR etc. ) of 100 with any other devices like in this case 200. 
250 is the similar means of connection. 
Both 100 and 200 are connected to each other - for which may 100 might be using any other port 130 and same for 200 where it might be using 230. 
Now when the user runs device 100 and runs application 110, his data related to that state is collected and transferred to the other device 200 over the connection. This transfer is depicted as 160. 
This data is stored by the runtime 220 as 111, 112, 113, 114, 115. 

Now runtime selects the compatible application 210 which may have it's own session 211 and other related data. 
Now runtime 220 uses 111, 112, 113, 114, 115 to determine what can be populated in 210 , which may include the text or any info the user had entered in device 100 and this allows to run the application 210 with the user's data. 
Now the user uses device 200 to continue using editing or working on his data using app 210 and once he finishes, runtime 220 collects the modified data from the app 210 and sends it back to device 100 , depicted in 260 so that in 100 the runtime 120 uses that to populate the app 110 and within the same session as 111 and if the need is to update some server over the internet, it can do so via using the same session and thread via the same network port as 140. 

Note : The above example shows only a generic approach and describes the overall components. Depending on the scenario the usage might vary slightly -- for example if the user does not need to use same session and the port to connect to server, the flow might be different where once the user has updated his info in 210 on device 200, instead of sending back the updated data to device 100 as depicted in 260, the 220 runtimes can submit it to server or backend using 240 over a different session.

Note: The above example shows only a generic approach and describes the overall components. Depending on the scenario the usage might vary slightly — for example if the user does not need to use the same session and the port to connect to server, the flow might be different where once the user has updated his info in 210 on device 200, instead of sending back the updated data to device 100 as depicted in 260, the 220 runtimes can submit it to server or backend using 240 over a different session.

Why is Omnipresence is the future for OS?

Interestingly, when Cloud OS concept came into existence, the driving force behind it was the thoayeught that a ‘good’ OS gets out of the way and lets the user get straight to what they want. In Microsoft’s approach – From the perspective of the user, they’re no longer using a program on a machine but consuming a service that lives in an arbitrary place. Because the service runs on an API common to all machines, it becomes easier to scale and failover. This is concept the drives Platform as a Service (Azure’s tour de force). [Source]

Big players like Microsoft also believe that the “perfect future [of OS] would mean that […] software is totally portable between desktop and mobile devices. “. The kind of attempts by such players are to converge the OS of mobile as well Desktops.

Though many attempted in the past to evaluate thin-client OS as the future (e.g. ), also gradually started to foresee the future of OS as something that can not be seen in the light of only a server-client architecture rather it would be defined in something more organic that would be sustainable. Articles like Desktop 2.0 and the future of the networked operating system , interestingly long back predicted the conclusion that we are carrying forward today that the Chrome OS version of the future where all we need is a browser is wrong.

So how Omni-present OS concept is different than thin-client and (even some old geeks referring the Mainframes to get the secret-sauce for the future!)? Here are the basic differentiators:

The Begining:

As I mentioned, in today’s OS evolution, it is the diversification of operating systems that prompt us to pause and think about the missing bridge to the future. If we closely ponder, it appears that the missing piece is convergence. The Omnipresent OS concept is a thought towards that direction. It’s actually a beginning for us to be prepared for tomorrow’s decentralized networked world where the age-old philosophies will show us the way!

(c) 2017-18 Samir Dash, Attribution-NonCommercial-NoDerivs 3.0 Unported (CC BY-NC-ND 3.0)


Beta Testing in the Ever-Changing World of Automation

Beta testing is fundamentally all about the testing of a product performed by real users in a real environment…Beta testing relies on the popular belief that goodness will prevail, which defines the typical tools that carry out such tests. Examples are the shortening of beta cycles, reducing the time investment, increasing tester participation, improving the feedback loop and visibility, etc…we lack “smartness” and proper “automation” in Beta Testing Solutions…

Check out the complete article at RedHat Developers Blog:

When Your Style Protects Your Data!

During the end of Summertime in 2013, while scanning through the news sites, one bumped upon an article about the vulnerability of the PIN we use in mobile phones to secure our personal data “PIN-Punching Robot Can Crack Your Phone’s Security Code In Less Than 24 Hours”. This made me explore more about the different approaches and mechanism employed across the smartphone and devices industry to loack /unlock the home screen. I came across the different workflows including the swipe to unlock, 3D gesture of hand infront of the camera of the device, biometric approaches (e.g. face recognition, fingerprinting ) along with smart ways like the “TimePIN” and Apple’s gesture based swipe etc.


In almost all the options, the device was stationary and the user’s feature was in action or in motion for example the swipe to unlock uses the finger movement, where as the sign wave by apple was more geared towards the gesture made by the user and similar.

So I kept on exploring about the concept and started to play around the concept of locking and unlocking from the other angle. A few days prior to this while watching a movie of the Tamil Superstar Rajinikanth’s cult style of wearing a glass I was reminded about one of the key insights from a local dipstick study on the local user segments for a popular Korean mobile brand which was about the fact that the emerging market mass has the urge to maintain the identity with the bragging power or the “massclusivity


Massclusivity is the term coined around “Masstige” which is a popular trend now a days that is employed in mass-market products infused with prestige elements (think smartly packaged USD 20 shampoo sold in supermarkets), catering to emerging middle classes with an appetite for premium, but budgets that do not stretch to true luxury. This gives the user, in the mass, the exclusiveness of his sense of fulfilment of his identity among the mass, through some kind of fashion statement.


Looking at this aspect for smart phones, which are more personal to the user than any other device, I started toying with the ideas of how can we make the locking/unlocking a fashion statement.

The first step towards this was to give a fresh facelift to the user’s interaction point with device i.e. the moment he enters the data to lock or unlock the device. To move out of the confinement of traditional thinking around the interaction of locking or unlocking the device, I tried to see from the other angle, where the device instead of being stationary, is moving along with or in respect to the user’s focal point. This helped to elevate the user’s perception of being in a position to enter the data to something more fashionable, a style statement built with some movement motion and energy associated with. For this I sarted with the basic use of gyroscopic and magneto meter’s readings. The initial base concept was just changing of the values in device orientation and/or geographical /geological direction, while a pass code is entered.

The user interface provides an indicator for a number of taps/entries to be entered sequentially as a passcode.

The UI will have a button (B) that on accepting a tap/click from the user collects the values from system sensors related to “geological – direction”(D) and orientation data (O).

There must be an option in the settings while the user sets the passcode that allows users to instruct the system if it will record both the values G & O or just either one of them to form the final passcode to be matched for unlocking the device.

While this seemed practical enough, I wanted to take the next step, towards the style statement. So instead of moving a finger on the device, or making a hand gesture in fromt of the stationary device, I tried to flip the components of interaction so tha the device would be in motion and make the necessry style stament through the gesture recorded by the the orientation and directional data change through the sensors. The user can over the air can make some gesture while holding the device, that would help making a style sttament.

In this approach, basically, the user will move the device and while in pre-determined device orientation & geographical direction taps the button (B). A consecutive number of taps as required triggers the complete sequence to be matched with the previously stored passcode. If the match happens then the device is unlocked.

The user can provide a threshold of tolerance for accuracy error to be used while setting up the passcode. For example, the user can set the threshold for tolerance of accuracy error to 0 then the device will require exact value match.

If the user has set the threshold of accuracy error to a moderate level, then if the recorded value falls with-in the accepted range then the match happens.

Enhanced implementation (Approach 2) was another step forward that used a motion in the air to make a shape of a sign using the device as a statement that would be used to lock and unlocking the device. For example the user can make a shape of heart on the air while holding the device.

The great part of this was that this would act as a perfect USP for wearables like smart watch and fitness devices.

In all the discussed approches basically the saving and matching the saved signature or signature are the 2 steps.


Saving a path in 3D space as a passcode:

In this alternative implementation, the user moves the device in to draw a pattern over the air in physical world 3d space.

The system automatically records the consecutive device orientation values at different intervals during the movement to draw the path in virtual 3D space and uses that as a pattern.


Unlocking Device

When next time the user moves the device in physical 3d space to draw the same path holding the locked device, the device matches this new path with the earlier saved path and if it matches then unlocks the device.


1.The device may map only the resulting path and not the exact co-ordinate and if similarity between the saved and user provided paths is more than certain threshold (similar to how we match signatures) then it can allow to unlock the device.

2.The saving of path can also be recorded along with the geographical direction so that only the user facing the certain direction should attempt unlocking. This can be higher level of security.

In Smartwatch also the user can move the watch to record a path as a passcode and provide the same to unlock the watch without even the need to look at it  –

In the small display area in devices like smartwatches, the invention becomes a boon as the user does not need to enter anything in the small UI

It’s always the fun and exciting to rediscover what we think we have already discovered. In case of this little idea, I feel that the excitement was the same and subject was worth re-discovering.

Re-imagining Beta Testing in the Ever-Changing World of Automation.

Based on the paper on the Beta Testing presented at RedHat QE CampX , Bangalore on 7 Dec 2017

Beta-Testing fundamentally is all about ‘A test of a product performed by real users in the real environment’ — and of course, there are many tags we use to refer to the testing of similar characteristics like User Acceptance Testing (UAT), Customer Acceptance Testing (CAT), Customer Validation and Field Testing (more popular in Europe) etc. Whatever the tag we use to represent all these testing cases, the basic components are more or less same — it involves the user testing and front-end UI as well as UX related testing to find out potential issues and rectify. Also, this always happens in the iteration of SDLC where the idea has transformed into a design and has passed the development phases where mostly the unit testing and integration testings have already happened.

Basically, the Beta stage of the Product Life-cycle Management (PLM), is the perfect opportunity to hear from the target market and plan for the road ahead.  When we zoomed into this phase of testing it has a board range of spectrum that ranges from Front-end or UI related testing involving UI functionality, Cosmetic, UI level Interaction and Visual look and feel at one end to the User Experience (UX) at the other end including User Testing involving more from A/B (Split) Testing , Hypothesis, User Behaviour tracking and analysis, Heatmaps, user flows and segment preference study or Exploratory testing and Feedback loops.

Beta Testing has the popular beliefs on the dimension of goodness that defines the typical tools that enable the user to carry out such tests, namely — Shortening of beta cycles, reduce time investment, increase tester participation, improve feedback loop and the visibility etc. However, if we dive deep into the factors behind the existence of tools from different angles, we will find two major reasons that advocate the importance of Beta Testing.

1. Left-Right Brain analogy that points out to the overlap of Human and Technology.

The typical belief is that the left-hand side of the brain mostly processes the logical thinking whereas the right part is more about emotional parts of the thoughts. Based on this popular analogy, when we map the different aspects involved in different stages of SDLC for a digital product across a straight-line from left to right, (refer to the diagram below) we will notice the logical and more human-centered aspects are divided by an imaginary line from the center. We will also notice the gradual progression of the emotional index for the components from left to right.

And when we map these to the beta testing phase, we notice these right-hand components are predominant in such testing phases. As users, as the humans, of the products, we are more emotionally connected to such aspects of the product which are validated or verified in a Beta Testing, thereby making Beta Testing as one of the most important testing phases in any SDLC.

Another interesting point to note here that when we look from the traditional software approach to define “criticality”, the areas that are tested during UAT / Beta, mostly fall into Class 3 and 4 type of criticality. But because these touch the core human aspects, they become more important.

To illustrate this here is a nice video reflects the importance of technology touching the human emotions. This YouTube video posted was a popular brand that offers a glass for the color-blind people which can correct the colorblind vision issue in the real time for the end user. Interestingly this aspect is about “Accessibility” that is one of the aspects that is typically covered during a Beta Testing. Just by looking at this aspect “Accessibility”, in context to the video, naturally, the question comes “What can we do for this father and the son, as a tester or a developer or a designer?”. And when we look at the stats, we find the number of people the accessibility impacts are huge– Every one in five-person is challenged by some kind of disability. But unfortunately in some reports indicate that at more than 90% of the websites in 2011, were not conformant to W3C’s accessibility guidelines.

This itself shows the human angle, that advocates why Beta Testing is important to ensure these aspects are validated and verified so that the target user needs are fulfilled and not go unattended.

2. From the standard’s perspective – Evaluating from ISO/IEC 9126-4 (2001) dimension that defines the difference between usability and quality as a perceptive line.

The International Standards Organization (ISO) has been getting the standards around Quality vs. Usability evolved over time. During the 1998 ISO identified the efficiency, effectiveness, and satisfaction as major attributes of usability. Just a year after that, in 1999, the quality model proposed involved an approach to measure quality in the light of internal software quality and external factors. In 2001 the ISO/IEC 9126-4 standard suggested that the difference between usability and the quality in use is a matter of context of use. The ISO/IEC 9126-4 (2001) standard also distinguished external versus internal quality and defined related metrics. Metrics for external quality can be obtained only by executing the software product in the system environment for which the product is intended.   This shows that without Usability / HCI in the right context, the Quality process is incomplete. One interesting point to notice here that that “context” referred here is actually that is fundamental to a beta testing i.e. “real users in a real environment”, thereby making the case of Beta Testing stronger.

Now we know why the Beta testing is important, let’s explore the challenges that are involved with the Beta stage.  If we notice any standards defined, including ISO/IEC 9126, most of these are static — none of these models really describe the relationship between phases in the product development cycle and appropriate usability measures at specific project milestone. We should note that any standard also gives relatively few guidelines about how to interpret scores from specific usability metrics. And specific to usability as a quality factor, it is worth to note that usability is that aspect of quality where the metrics have to be interpreted.

In this light when we look at popular Beta-Testing tools of today, we can notice that the Top Beta Testing tools leave the interpretation to the customer’s or end user’s discretion. This brings to our Number one challenge in Beta Testing — How to filter-out pure perception from the actual & valid issues and concerns?   As most of the issues are related to user-testing, split-testing, and front-end testing, there is no optimized single window solution that is smart enough to handle this in an effective way.  Real users in the real environment are unless empowered are handicapped to comprehend all the aspects of beta testing and react. Also, it’s all perspective and all of them cannot be validated with real data from some benchmark/standards.

The World Quality Report in 2015-16 Edition indicated that the expectations from the Beta testing is changing dramatically. It hinted that the customers are looking for more product insights through a reliable way to test quality, functionality along with the regular usability and user testing in real customer facing environment.

It’s not only the Beta Testing rather in the overall testing scenario, more user-demand is also getting impacted by the rising complexities and the challenges which is increasing due to accelerated changes in the technology, development and delivery mechanisms and approaches. The 2017-18 World Quality Report reports that the test environment and test data continue to be Achilles heel for QA and Testing along with the fact that the challenges with testing in agile development are increasing. There is now a demand for automation, mobility, and ubiquity along with smartness to be implemented in the software quality testing. Many believe that the analytics-based automation solutions would be the first step in transforming to smarter QA and smarter test automation

While this true for overall QA and testing, this is also true for Beta Testing, even when this testing, unlike the unit testing, system testing etc. deal with the functional aspect of the product.

Let’s see where we stand today as per this benchmark. If we explore popular beta testing solutions, we will get a big vacuum in the area where we try to benchmark user’s need for more functional aspects along with the usability and user testing aspects are mapped. Also, you can notice in the diagram that there is ample space to play around with the smart testing scenario with the use of cognitive, automation and machine learning.

(Note: Above figure shows my subjective analysis of the competitive scenario.)

Basically, we lack “Smartness” and proper “automation” in Beta Testing Solutions.

Apart from all these, there are some more challenges that we can notice if we start evaluating the user needs from the corresponding persona’s viewpoint. For example, even when assuming that the functional aspect is to be validated, the end-user or the customer may have an inability to recognize it. As the product -owner, customer or the end-user who are the “real users in a real environment”, are part of the user segment who may not be aware of the nuts and bolts of the technology involved in the development of the product they are testing to sign it off. It’s like the classic example of a customer who is about to buy a second-hand car and inspects the vehicle before making the payment. In most of the cases, he is paying the money without being able to recognize “What’s inside the bonnet!”. This is the ultimate use-case that advocates to “empower the customer”.

Now how to empower the end user or the customer? The tools should support that in a way so that the user can have his own peace of mind while validating the product. Unfortunately, many small tools who try to solve some of those little issues to empower the user (for example the Google Chrome extension that helps to analyze CSS and creating the report or an on-screen ruler that the user can use to check alignment etc.) are scattered. The ground reality is that there is no single-window extension/widgets based solution available. Also, not all widgets are connected. And those which are available, not all are comprehensible to the customer/end-user as almost all of them are either developer or tester centric. They are not meant for the customer without any special skills!


When we look at the automation solutions in testing as part of much Continous Integration (CI) and Continuous Delivery (CD), are engaged and effective in mostly “pre-beta” stage of SDLC. Also, they require specific skills to run them. With the focus on DevOps, in many cases, the CI-CD solutions are getting developed and integrated with the new age solutions looking at the rising complexities of technology stacks, languages, frameworks etc. But most of them are for the skilled dev or test specialists to use and execute them. And this is something that does not translate well when it comes to Beta testing where the end-user or the customer or the “real user in real environment” are involved.

Apart from all these even, assuming we can have all these automation features enabled in BETA it still points to another limitation in the existing solutions. It’s because the employment of automation brings in its own challenge of “information explosion” , where the end user needs to deal with the higher volume of data from automation and with so much information, the user will struggle to get a consolidated and meaningful insight of the specific product context.   So what we need to solve these challenge? Here is my view — We need smart, connected, single window beta testing solution with automation that can be comprehensible to the end-users in a real environment without the help of the geeks.

For sometime since a last few years, I have been exploring these aspects for the ideal Beta Testing solution and was working on the model and a proof of concept called “Beta Studio”, representing the ideal beta testing solution, which should have all these — Beta-Testing that utilizes data from all stages of SDLC and PLM along with the standards + specs and user testing data to provide more meaningful insights to the customer.  Test real application in real environment by the real users. Customer as well as end-user centric. Test soft aspects of the application — Usability, Accessibility, Cosmetic etc.  Smart enough to compare and analyze these soft aspects of the application against functional testing data.

Use machine-learning & cognitive to make the more meaningful recommendation and not just dump info about bugs and potential issues.

Here is an indicative vision of Beta Studio:


Mostly this vision of the ideal beta testing solution touches upon all the aspect we just discussed. It also touches upon all the interaction points of the different persona e.g. customer, end-user, developer, tester, product owner, project manager, support engineer etc. across the whole Product Life Cycle and utilizes automation along with the Machine Learning components such as Computer Vision (CV) and Natural Language Processing (NLP) to gather information that has to be processed by the cognitive aspect to generate the desired insights about the product and recommendations. During this process, the system will involve data from standards and specs along with the design benchmark generated from the inputs at the design phase of the SDLC, so that meaningful insights can be generated.

In order to achieve this vision being translated into reality, what is that we need. The following diagram hints exactly on the same:

Basically, the first step should involve creating the design benchmark from the information from the design stage, that can be used in comparing the product features against a metrics based on this design benchmark.

The second thing that matters should be automated and manual tracking of the product during runtime in real time and categorize and collate these data.  The third step involves creating features to support the user feedback cycle and user testing aspects (exploratory, split testing capabilities).

The fourth step would be to collect all standards and specifications on different aspects — e.g. Web Content Accessibility Guideline (WCAG) Section 508, Web Accessibility Initiative Specs ARIA, Design Principles, W3C Compliance, JS Standards, CSS Standards & Grids, Code Optimization Metrics Error codes & Specs, Device Specific Guidelines (e.g. Apple Human Interface Guideline) etc.

The fifth is about building the critical components such as Computer Vision and Natural Language Processing units which would process all the data collected in all these stages and generate the desired metrics and inferences.

The sixth step involves building the unit to generate the model to map the data and compare against the metrics.  The final or the seventh step is to build the cognitive unit that can compare the data and apply the correct models and metrics to carry out the filtering of the data and generate the insights which can be shared as actionable output to the end-user/customer.

While experimenting for BetaStudio, I have explored different aspects and built some bare bone POCs. For example, Specstra is a component that can help create Design benchmark from design files.

When I was exploring Specstra , that was trying to address the issues related to the Cosmetic aspect or visual look and feel. Typically when it comes to cosmetic issues, more than 30% or one-third issue are non-functional and mostly Cosmetic, there is no reliable solution that helps in benchmarking this kind of issues against some specific standards. At a minimum, one third of the issues found during the beta / UAT stages of testing are mostly cosmetic and alignment issues including category 3 and 4 types. And these happen mostly because the two personae – developer and designer involved have their own boundaries defined by a mythical fine line.

When a developer is in focus roughly 45% of them do not aware of all the design principles employed or the heuristics of UX to be employed. Similarly half of the designers are not aware of more than half of the evolving technological solutions about design.

And in mostly three fourth of the of the projects, we do not get detailed design specs to benchmark with. Detailing out a spec for design comes with a cost and skills. In more than two-thirds of the cases of development there is the absence of detailed design with specs. Wherever it exists many of the designs are not standardized, also most of them do not have clear and detailed specs. Also being design is carried out by different tools it is not always easy to have a centralized place where all the designs info is available for benchmarking.

To solve this Specstra comes handy as it is an Automation POC that is more like a cloud-based Visual Design Style Guide Generator from the third party design source files – this is a case where the user would like to continue using his existing Design tools like Photoshop/Sketch or Illustrator, PDF etc.

You can view the video of the demo here or read more here or here

Similarly, the single window solution of tracking and be getting the real-time data on accessibility, visual design, CSS, JavaScript & Environment etc. from the product is explored in this POC. View a video on Beta Studio POC here


I know reaching the goal of an ideal beta testing solution might need the effort and time and the concept will also evolve over time. But for sure the journey has started for all of us to connect and explore how to make it a reality.

Feel free to ping me in the comment section of this article.

To explore the open-source project of BetaStudio poc , follow the link here – (will be uploading all the code to the repos in these in coming times. )