Observe is a robust observational assessment tool that was designed primarily for high-stake clinical skills assessment, such as Objective Structured Clinical Examinations (OSCE, pronounced OSKEE) and Multi-Mini-Interviews (MMIs) for admission tests. These tools use video integration, making it possible to have a remote face-to-face interaction between the interviewers/examiners and the candidates. Besides data collection and recording of results, Observe also has features that enable psychometric analysis, multi-examiner / multi-location, and direct feedback delivery.
If you want to conduct clinical skills exams or interviews, tools like Google meets, MS Teams or Zoom may not be effective because they lack some of the functionalities that are crucial for this kind of assessment. Observe has features that are tailored for medical, nursing, dentistry, and veterinary OSCEs and Multi-Mini-Interviews (MMIs) alongside other clinical exams/interviews.
For more precise results, this tool uses the methodological principles typically for OSCEs and MMIs to enable brief consecutive (10 to 15 minutes) stations for assessing the different clinical tasks. Medical schools and institutions can use this tool to cut costs incurred in interviewing candidates remotely.
Our client tasked us with building the remote video parts of the examination analysis of the software. The main application of this tool was to make the remote assessment of medical students as seamless as possible and fully compatible with the on-site examinations. So, we had to build a tool that examiners, simulated patients/actors, and candidates can easily use without going through a lot of hassle.
After signing up as a student, one is given access to a list of exams that they can select based on their course. Students also have the option of customizing the video before the exam and positioning it in the platform's UI in a way that makes them comfortable. Students can also communicate with the actors before the exam.
The examiner's role in this tool is to observe the students as they take the exams and follow all the set rules. The user interface of the examiner's dashboard has tools that make it easy and convenient for the examiner to monitor the students as they take their exams.
When the actor visits the site, they have the option to see the list of exams they are assigned to, customize the video, and also view instructions for each exam.
Unlike the other users, the administrator has much more access to the backend of this system. Their dashboard has the option of creating and configuring the exam, adding media, changing stages, and the ability to report any issues. The administrator can also join any role if there is a problem.
We implemented the Keycloak user authorization management tool: This feature enables the different users on this platform to create accounts and become members. The type of account the users create determines the level of access they have to the system.
SSE connection: This feature enables a user to get information about any changes made in the system by other users.
Administrators now have the option of enabling feedback entry using an assessment form section. This is to allow examiners to give their comments for the various assessment questions of the exam.
In the new design, there is an option of attaching extra instructions to the student feedback reports for all or “fail” grades. This feature also allows the user to include a digital signature.
With the new media center, users can now upload their files in bulk and attach them to their exam entries like stations and scoresheets with different access levels based on their account type. Some of the supported files include mp4, pdf, and jpg.
The new remote assessment feature allows exam coordinators to facilitate the remote assessment of students. This has been made possible through video integration and exam logistics automation. This new feature through the watch tab also gives the coordinator the option to monitor remote stations and communicate with various participants (students, actors, or examiners). Users can also send custom notifications to a specific group of participants, notifying them of the next action or providing them with additional user-related information.
The new design also includes an upgrade to the assessment component to support remote assessment. It also allows receiving exam status changes, prompts, and requesting assistance from the exam coordinator/admin whenever the need arises.
This new design includes a backend API for Observe that integrates it with any platforms that would like to take advantage of its features.
Additionally, a console app is included in this new design: This allows students and simulated patients to access a new dashboard that enables them to participate in remote examination sessions that support video integration.
With this new design, students and simulated patients can now preview an exam schedule and configure their microphone and camera before the exam. There is also an option to test network quality to ensure the network is stable during the assessment session.
Students and simulated patients have the "help" option in their dashboard. This option enables them to request assistance from the exam coordinator whenever they face a challenge while using the system.
The top features created by us
Integration with Twilio:
We integrated the Twilio API into Observe to enable sending and receiving texts and other communication functionalities. This feature creates tokens and disconnects/connects and also creates backend commands.
Having the instructions rendered was very crucial because these are what the students had to follow while being assessed. Each of the students using Observer has instructions to follow while using this software. All documents that contain instructions that students have to follow are stored on the s3 bucket. Users can download more videos and attach them to an additional media center like a Vikorist Moxie manager. Moxie manager - file display is connected to the s3 bucket and can connect media between students and examiner.
Control and tracking of problems by the exam by the admin
This feature enabled administrators of the platform to track all the issues that other users faced while using the platform. It also allowed the admins to have complete control of the other users to ensure they operate within the assessment guidelines.
Integration with Keycloak
This feature was crucial because it enabled the different users to create accounts that determined the level of access and control they would have while using Observe.
Integration with moxiemanager
We added moxiemanager to the system to enable seamless management of files like images, PDFs, and any other form of documents.
Rating analysis and feedback generation which takes place in PHP
Testing was done on the client’s side
The client chose Amazon Website Services to host this platform because of its well-built infrastructure with a good balance of performance and security.
Methodology of development
2020 - Current time
1 Front-end, 2 Back-end developers