With the proliferation of Customer Data Platforms, it has become even more important that your quality analyst do a much better job ! One miss on validating the identity fields within the analytics calls and all (all is an exaggeration ! ) the well curated profiles would go for a toss, stitching multiple users (technical parlance — profile collapse).
The above piece is just one use case (more on the CDP piece later this week) to give you an idea about the importance of robust QA process.
I had created a python script a few years back which helped automate the process of QA for the team. Below, I share the overall framework on how to go about it. I had shared this as a Linkedin post a few years back but i think medium would also be a good ‘medium’ to get this across to more folks.
Where it all started
I remember scouring the internet to come up with ‘the’ panacea for my QA fears. I stumbled upon this brilliant article from John Simmons https://medium.com/@johndavidsimmons/testing-adobe-analytics-with-python-368752a39cc2 which made me believe in the power of google and my own will to make it happen. I had to automate my way out of my misery.
The above article was a great starting point to my pursuit. It talked about how we could use Python and our old friend selenium to grab the Adobe Analytics calls from the console of the browser. I was so excited with the discovery that i immediately got working on my own solution with tweaks.
The above article didn’t talk about how would you go about making it generic and create a setup which would automate the entire QA process for the website. Not just one website but any website you would like to QA.
Second limitation it had was that it relied on the console to grab values for the analytics calls.
Third limitation was that it was specific to Chrome browser. I had to make it browser agnostic.
Fourth limitation was that it only worked for Adobe Analytics.
So the above approach was neither browser agnostic nor analytics package agnostic. Nevertheless, it was an awesome start making me hopeful of light at the end of the tunnel ! Yeah, i know ! Exaggeration ! Puff !
I then set out to get the above approach sorted to make my solution browser agnostic (nothing can help Internet Explorer but for the rest of the folks out there ! ) and package agnostic (i needed to QA so much more than just Adobe Analytics calls). I also needed to make it generic enough to work on any website and to be able to multiple regression tests at once for multiple journeys.
My Approach
I was determined not to spend rest of my week in front of my screen looking at every network call to verify the data being captured clicking through pages and multiple journeys.
After many iterations, following approach produced results for me:
1. Excel based flow: I made provision for defining multiple flows using excel sheets. Every sheet represented one journey. This way, i could define multiple journeys in one go and then just run the script.
2. Read the flow and feed that into selenium: Once the flow was defined, the script read the flow defined in the excel and fed that elements into selenium driver. This would help iterate through the journeys.
3. Use a proxy server to route the request through: My approach was centered around using a proxy server and passing all the network packets through it. Next step for me was to grab these network calls and save them in a HAR (Http ARchive) file.
4. Parse the HAR JSON: Next step involved parsing the HAR file JSON and putting the desired network calls into another excel file. Et Voila ! Your adobe analytics parameters are right there in front of you in that excel well formatted ready to be mailed across.
5. Validation of results (Optional): Once we have the network calls in the excel, and we know the expected values (if we do) then we can compare the two and come up with a validation sheet as well! This step is optional as many a times, we might be capturing multiple possible values in the variables and putting those down to validate can itself be a project.
This is still a work in progress and i would like to build further upon it when time permits.
Let me know your thoughts about the process and also the article in the comments sections.
Leave a Reply