Last week I served as a Subject Matter Expert for Amazon Web Services to create test questions for the Solutions Architect Associate Exam. I’ve now served on two of these IDW’s and I thought I’d share my experience.
My journey to become a AWS Subject Matter Expert (SME) actually began in April 2018, when I visited the Certified Lounge at the AWS Summit in San Francisco. In the lounge, there was somebody scanning badges for people to volunteer to become SME’s. I figured I’d throw my name in and see what happens.
Almost a full year later, I received an email from AWS on the next steps to become a SME. Those steps was a survey, first asking which AWS Certifications I held, then which products I used, in which capacity, and to provide some examples of my work.
Once I completed that very long survey, I figured I’d never hear from them again … but I did, a little over 6 weeks later! I was being invited to my first Item Development Workshop to be held at the AWS office in Chicago (how convenient!) in June.
Before I could attend my first workshop though, I was required to attend the AWS Subject Matter Expert Item Writing course online and pass the exam with a score over 92%. Let’s just say … it took me a little longer than the noted 2.5 hours, but I somehow got through.
Last week I attended my second Item Development Workshop was 3 days long and the formats were very similar to each other. The sessions themselves were attended by approximately 12 individuals, with about 8 of them being AWS employees, and 4 being external experts. After the introductions, each facilitator told us how our exam writing ends up on the actual exam itself.
We were given the task to write exam questions for particular exams with expanded exam blueprints, which are not publicly available.
The first and second days were split pretty equally on exam writing and group reviews. Once we wrote exam questions, we were split up into two groups to review questions. The first group was given questions to review questions written by individuals from the second group and vice versa. The third day was spent mainly doing group reviews of questions and tying up loose ends before passing along to the next stage.
The group review was equally as difficult process as the question writing as well. Each question was projected on a TV and we discussed what we thought the correct answer was. There were several times when others believed an incorrect answer was correct or the the opposite. (This is when stuff gets heated!) Usually this required us to spend more time than the facilitator allocated for us on the question and we poured over user guides, FAQs, and white papers to provide evidence. If the answer or incorrect answers needed to be changed, those questions were assigned to somebody else in the room for further clarification.
After each of my IDW’s, we completed anywhere from 70-90+ questions for future exams. After we wrote the questions, they went on to a Cloud Technical Architect for review of accuracy, then went on to an Alpha Testing, which is an internal test with other Subject Matter Experts to determine to viability of the question on the exam. If at any point, it’s determined that the question or answer isn’t sufficiently clear or just tested poorly, the question is sent back to the pool for further modification or is scrapped from the exam all together. After the Alpha Review phase, questions begin to flow into the actual exam.
All in all, it’s a pretty grueling process to write exam questions, review other questions with your peers, and then hope they show up on the actual exam. In an effort to maintain the integrity of the exam, we’re never informed if our questions showed up on exams, but I like to believe they eventually will.
That being said, I do recommend trying to become a SME for AWS. It’s a fun experience and a bit of a break from your day to day. You’ll get to learn about products you’d never used before and get to work with other great SMEs across the world.
If you’d like to join the AWS SME, fill out the form located here.
