The Importance of Safety in the Utilities Industry: A Q&A with PG&E

Headshot Julie

Posted by Julie Delazyn

Wendy Lau is a Psychometrician at Pacific Gas and Electric Company (PG&E). She will be leading a discussion at Questionmark Conference 2016 in Miami, about Safety and the Utilities Industry: Why Assessments Matter.

WendyLau_Q&A

Wendy Lau, Psychometrician, PG&E

Wendy’s session will describe a day in the life of a psychometrician in the utilities industry. It will explore the role assessments play at PG&E, and how Questionmark has helped the company focus on safety and train its employees.

I recently asked her about her session:

Tell me about PG&E and its use of assessments:

PG&E is a utilities company that provides natural gas and electricity to most of the northern two-thirds of California. Over the years, we have evolved into a more data-driven company, and Questionmark has been a part of that for the past 7 years. Having assessments readily available and secured within a platform that we can trust is very important to PG&E. We are also glad to have found a testing tool that offers such a wide variety of question types.

Why is safety important in the utilities industry?

Depending on the activity that our employees perform, most of the work has serious safety implications — whether it is a lineman climbing up a  pole to perform liveline work or a utility worker digging near a major gas pipeline. Our technical training must have safety in mind and, more importantly, it must ensure that after going through training, employees are competent to perform their tasks safely and proficiently. In order to ensure workforce capability, we rely heavily on testing to prove that our workforce is in fact safe and proficient and that the community we serve and our employees are safe and receiving reliable services.

What role does Questionmark play in ensuring that safety?

Questionmark helps us focus on safety-related questions by allowing special assessment strategies such as identifying critical versus coachable assessment items and identifying cutscores for each accordingly. Questionmark also allows a secured platform so that we can ensure our test items are never compromised and that our employees are truly being assessed under fair circumstances.

To find out more about the role of Questionmark plays in ensuring safety, you’ll just have to attend my session at Questionmark Conference 2016 in Miami!

What are you looking forward to at the conference?

I am very much looking forward to ‘talking shop’ with other Psychometricians and sharing best practices with others in the utilities industry and other companies alike!

Thank you, Wendy for taking time out of your busy schedules to discuss your session with us!

palm tree emoji 2If you have not already done so, you still have a chance to attend this important learning event. Click here to register.

 

Performance testing is to certifications as simulation is to learning

Howard Eisenberg HeadshotPost by Howard Eisenberg

I just attended the Performance Testing Council Summit. Performance testing is “testing by doing.”  Exam developers create performance items that require candidates to actually perform real-world, authentic task not multiple-choice questions that have only one best answer or allow a low-ability candidate to guess the correct answer.  The outcome of the task is then evaluated to determine a score, or how well the candidate performed.

All but one of the attendees of this meeting representing certification programs were from software/IT companies. The IT domain lends itself very well to the adoption of performance testing. Advances in virtualization and software-as-a-service make it possible to provision “testing labs” with specific characteristics and traits in minutes and at low costs.  Moreover, as these labs can be hosted in the cloud nowadays, there’s no need for a candidate to travel to a specific location to take an exam.  This means that IT, performance-based certifications can and indeed ARE being delivered online and on-demand, with the help of remote proctoring and other technology-enabled security controls.

sunWe’d like to hear from you! The call for proposals is officially open for the Questionmark 2016 Users Conference. If you have an experience you would like to share with the Questionmark community, please submit a presentation proposal here: click here.

These labs are the real-world context in which an IT professional works.  It’s not a simulation of the software, tools, network connections, etc.: it is the real thing.  As such, it’s arguably a more valid methodology for assessing an IT professional’s ability to perform the tasks required by the job.  So using a performance exam for an IT certification makes sense.

Alas, not all certification exams and the professional domains they represent are as well suited to performance testing. It’s not as easy to recreate the environment in which a registered nurse performs his or her daily duties, for example.  In other domains where technology is not center-stage, Questionmark’s customers have historically done the next best thing.  What’s that, you ask?  Well, it’s simulating the performance environment within the test. And if high-fidelity simulation is not cost-effective to develop, then it’s using real-world exhibits, artifacts, and scenarios expressed through multimedia to bring as much of the performance context/environment to the test as is feasible and cost-effective.

Performance testing is to certifications as simulation is to learning.  It’s that “holy grail.”  If we can make the exam look and feel like the job, then it will have the greatest potential to be the truest measure of ability.  If we can make the training look and feel like the job, then it will have the greatest potential to adequately prepare the employee.  (I say “potential” only because the instrument or the simulation must still be well-designed).

I know that many Questionmark customers have struggled to attain this ideal. That is the reality of working with budgets, timelines and other limited resources.  But I’m willing to bet that many customers have creatively worked around these challenges to create valid tests and exams that provide solid measurement value to the programs in which they are used.

sunIf you have a story to tell about such challenges and solutions, then please share them with the Questionmark community at the Questionmark 2016 Users Conference. Click here to submit your presentation proposal. *Submission deadline is December 4. Slots are limited.