FRAMEWORK FOR EXTENDED TRANSCRIPTS: PILOT IMPLEMENTATION & RESULTS (APRIL 2017)

Authors: Joellen Evernham Shendy, Associate Vice Provost & Registrar, University of Maryland University College, and Insiya Bream, Assistant Vice Provost, Registrar Strategic Operations, University of Maryland University College

At University of Maryland University College (UMUC), a New Learning Model project is underway that focuses on building a student’s competencies over time through project-based learning—providing closer connections between the learning and expectations of employers in the field. In the fall of 2016, five of UMUC's Graduate School programs, including the Master of Business Administration and some Cybersecurity programs, implemented the new learning model. As part of innovating and developing a new learning model, UMUC realized that the documentation and evidence given to students regarding their learning would need to change as well. Telling a student story, via a new learning model, requires retooling in order to share the story effectively. For years institutions have heard from employers and others that the universally issued student transcript does a poor job of conveying what a student knows and can do, and is insufficient in providing evidence that helps an employer and others determine skills, competencies, and potential fit for an organization. 

UMUC was one institution among 12 that received grant funding provided by the Lumina Foundation through the American Association of Collegiate Registrars and Admissions Officers (AACRAO) and National Association of Student Personnel Administrators (NASPA) Comprehensive Student Records Project. The project focused on piloting new comprehensive record models that display a student's educational and/or experiential background. These records would be in addition to, and not a replacement for, the original official transcript. The goal of the project was to identify models that could then be adopted by more schools—all helping to share the students' story in new and creative ways that speak to a more holistic view of who the student is and what they know. The launch of UMUC’s new learning model and the start of IMS Global’s Extended Transcript (eT) pilot were aligned and provided a natural segment of students that could give feedback on the first iteration of the Extended Transcript (eT) achievements record standard.

How We Did It

UMUC worked closely with Learning Objects through IMS Global’s CBE and Extended Transcript Workgroup, a part of the IMS Global Digital Credentials and Badges portfolio, to establish the initial data model that will allow institutions to manage and exchange competency and achievement data between systems, and ultimately, publish this data in an extended transcript achievements record store—for consumption by an employer, student, or other educational institution or organization. Through multiple iterations, Learning Objects, IMS Global members and staff, and UMUC’s IT team worked together to pull and display student competency data from the D2L learning management system (LMS). Learning Objects was able to digitally represent the competencies in a user interface (UI) that had a more modern look and feel, allowing the student to see, and interact with, evidence of their mastered learning.

Watch Learning Objects lightning talk from the 2016 Learning Impact Leadership Institute.

Pilot Details

An overarching goal of the eT pilot was to solicit student feedback on the concept of an eT achievements record and their initial impressions of the utility of an extended transcript. Through the pilot, we were hoping to discover specifics on the following:

• What did students think of the virtual document itself?
• Did students find the content useful?
• How would students use the information?
• Did they understand the purpose?
• How could a digital document like this help carry them towards their next professional goal?

Over the past five months, UMUC’s eT pilot students were able to access an eT achievements record displaying mastered competencies through an icon link in their D2L classroom. Through a "soft rollout," the link was activated and a week later students were notified via email of the pilot. The email included information on the overall purpose and asked them for their feedback via a survey by a link on the eT itself and in an email. One of the survey questions included whether or not the student would be willing to participate in a focus group in early spring 2017 to provide more detailed feedback on the eT. A “help” icon on the eT itself explained what UMUC hoped to achieve and learn through the pilot. The internal marketing team supplemented the email and eT with a web page that included general pilot information and an FAQ for the wider campus community. Out of approximately 2,000 pilot students that had access to the eT, over 800 students accessed the document. 

What do Students Think of the eT Achievements Record?!

UMUC students responded throughout the pilot period to the survey, sharing their thoughts and recommendations. The majority of survey respondents made it clear that they liked the graphical look and feel of the eT. Working with a third-party vendor, Learning Objects, allowed UMUC to help create the design and feel of the eT by utilizing a variety of input from various University stakeholders and by modifying the document until we were satisfied with the appearance and ease of use for students. Leveraging Learning Objects' technology brought a 21st Century user experience for students to the eT achievements record. 

Survey results showed that over 70% of respondents found the eT content useful. Students noted that they could use the information on the eT for resume writing to explain or communicate their competencies and achievements in the academic program. Students felt they would share this mostly with future employers, but also with other admissions offices, their current employers, family, and friends.  We asked survey respondents if they recommend we open this up to the larger UMUC community and the answer was an overwhelming YES.

UMUC also conducted a focus group to drill down into some of the survey questions to get additional information to help inform the next iteration of the eT. In general, the focus group students were positive about the eT and 84% recommended UMUC implement an eT-type achievement record for all students. They shared additional details about the appearance and content of the eT. Students felt that the eT would give them more confidence when pursuing job opportunities—helping them to articulate with clarity and confidence their competencies. Several students remarked that their own understanding of their achievements was influenced by the content they viewed in the eT record.  

The team of academic and technology colleagues working through IMS Global have been integral to the success of this effort.

"We extend hearty congratulations to UMUC for this critical first step connecting their students to employers through digital credentials. The Extended Transcript Achievements Record forms the foundation of an institution’s digital credentials strategy. UMUC and their peer leaders Capella University and the University of Wisconsin-Extension are paving the way to tomorrow’s marketplace of verified credentials."

—Mark Leuba, vice president,product management, IMS Global Learning Consortium

What areas of improvement did the students identify for the next version of the eT? Although most students did not feel the digital document was too lengthy, some felt the length would cause employers to gloss over the information. One idea shared was to allow students to curate and create their own “view” of the achievements record that was specific to a particular employer, organization, or industry. This would allow them to share a portion of, but not the entire, eT and highlight accomplishments most relevant to the students’ goal(s). Focus group participants noted that it would be interesting to see the competencies organized from an industry or sector perspective and to break down the “level” of competence based on position requirements such as entry level, middle, or C-Suite management.  There was also a student desire for the opportunity to contribute to the document by way of adding their own supporting content, providing them an option to make adjustments for their unique situation. 

Overall, the feedback collected through the survey showed that students understood the eT achievements record and would use it in multiple ways to reach their goals. While the respondents felt understanding their own accomplishments and articulating them were a huge benefit, they did feel most strongly that the connection with employers was critical. One particular comment from the survey highlights why we need to continue to build these bridges, via new ways to evidence learning, between institutions and the workplace:

“The document is a great reminder of what a graduate is capable of doing. Instead of saying I walked away from college with an x.xx GPA, I can say explicitly, this is what I am capable of doing; these efforts gained me these competencies.”

Future State of the Extended Transcript

UMUC successfully closed the pilot at the conclusion of the winter 2017 term. Our next steps are to leverage the work we are doing as an initial contributor to Credential Engine with adding our program competencies to the nationwide registry. This will provide additional linking and connection opportunities. UMUC is also working towards finding better ways to “match” a student's competencies with employer needs by continuing to work with innovative and dedicated colleagues through IMS Global in their digital credentials and badges program. As far as a next version of the extended transcript achievements record, we look forward to incorporating our students feedback and reaching out to the employer and recruiting community to build closer ties and feedback loops, ensuring our students will be armed with 21st Century digital evidence of their learning which presents a more holistic picture of their fit to organizational needs.

For more details on the next phase of the Extended Transcript, see Achievements Records Standard and SkillsCenter Search™

“Cause I’m not like everybody else.” —The Kinks

Standards are the foundation for diversity at scale. Just look around you . . . and then realize that all this diversity came from a common standard that we call DNA. Lack of a foundation in replicable standards means lack of scalability. Think the plethora of grant projects ending up in one-off successes.

Every teacher and every student are not like everybody else. The future of education isn’t massification; it’s personalization. Personalization at scale.

During IMS Global’s recent quarterly meeting in Orlando, we had the pleasure of hosting a Summit on digital micro-credentials, including Open Badges (a standard and community that IMS Global is now facilitating after a recent handoff from the Mozilla Foundation).

Micro-credentials and Open Badges are key foundations for scaling personalized education. They are a way to enable and recognize personalized learning and life pathways. In essence, they are a way to create and recognize each person’s unique story.

Educational systems and educators that embrace and perfect micro-credentials will be the ones leading us to the personalized and scalable future.

The Summit by itself would have made anyone’s week. But there was much more at the quarterly meeting.

Agreement was reached on a breakthrough competency and learning standards framework (Competencies and Academic Standards Exchange) that enables digital resources to be mapped to achievement frameworks and thus enables finding and selecting resources based on those frameworks.

A set of leading educational content providers established a breakthrough open source collaboration to enable ubiquitous availability of OneRoster®.

The emerging field of learning analytics continued its advance thanks to an incredible cadre of leaders in IMS Global who are making learning analytics practical via Caliper Analytics® and IMS Global’s vibrant partnership with ADL/xAPI.  

Among the most exciting sessions of the week were the Learning Tools Interoperability® (LTI®) Showcase and Hackathon. Suppliers and institutions are doing amazing things with LTI apps that can work across platforms. And, yes, there were some very tangible signs that the market is beginning to catch up with the more advanced features of LTI and LTI v2.

If you weren’t there and you’re not like everybody else come join us next time! You’ll see and lead the personalized scalable future of education and learning.

#7 and final post of a series in preparation for May 2016 Learning Impact Leadership Institute

In the previous posts of this series I have reviewed the requirements driving next generation digital learning environments, looked at the anatomy of interoperable educational apps and considered the realities of sector trends in learning platforms and products.  Episode #6 concluded that the evolution to NGDLE will be driven by a combination of high value use cases enabled by an interoperable product ecosystem.

The Critical Importance of Sector Leadership

First a bit about the sector leadership required in getting to the NGDLE.  From where I sit, education sector leaders need to decide if NGDLE (or whatever comes next) is something that is happening to us or something that we are going to make happen? 

To explain further, tech innovation in the consumer world tends to “happen to us.” Apple iTunes is not something that consumers invented.  If consumers had invented Apple iTunes I’m sure we would have chosen to make sure all those tunes and apps would work on any mobile device we chose, not just Apple devices. Or, why is the Apple library limited to use on 5 devices?  That is not very consumer friendly.  But, individual consumers don’t have much influence in terms of the platform strategies of high tech companies.

But, the education sector is different.  While historically there has been a steady pattern of what I call the "alien invasion" theory of edtech adoption, there is huge potential for institutions that can work together to shape the ecosystem, and therefore the platform strategies of suppliers, going forward.  Indeed, we are at a period in time right now where not only institutional leaders, but leading suppliers are largely agreeing that vendor-specific ecosystems are NOT the way forward in the education sector. No single vendor can provide the diversity of digital resources, tools, apps and platforms.  And no single vendor can provide the sales channel to dominate education revenue generation.

But, strong collaborative leadership is required to establish an open edtech ecosystem. The “system” of leadership that seems to be working are suppliers that can bring the product ideas and technical expertise while institutions bring the clarity of the high value use cases and the firm requirement for plug and play integration.   In IMS we have seen the combination of these two forces achieve some amazing things.  Leadership from both institutions and edtech suppliers in creating and evolving the ecosystem the NGDLE (as envisioned in the EDUCAUSE research and this blog series) is essential.

Use Cases and Emerging Product Categories that Will Shape NGDLEs

In IMS Global we definitely believe that necessity (aka compelling need and value proposition) is the mother of invention.  Exact timing of the cycle of adoption can highly variable based on a number of factors (see the discussion on filtering ideas based on the four major VC risk areas in the previous post of this series). In IMS we like to determine if the value proposition is strong, then establish a track of work and then let the market feedback help us sort out the risk factors and timing.

High value use cases for the NGDLE are those that enable flexibility, choice and personalization that lead to improved educational outcomes.  Here are some obvious winners:

• Seamless user experience for teachers and students: Sorry if this one seems rather uninspiring but the reality today is that the user experience in navigating among apps and platforms is still not very good.  Making the user experience seamless is key to enabling the development of NGDLEs that are seen as a step forward, rather than backwards. Indeed, an obvious first and essential step in building further confidence toward the NGDLE concept is to achieve seamless integration of some of the core instructional applications already in use at most institutions (see a list in the previous post).
• Curated digital resource choices with associated assessments: Student outcomes improvement ala Benjamin Bloom’s famous “2 Sigma Problem” is dependent on delivering effective differentiated instruction via a classroom cohort.  This infers well thought out instructional strategies and resources, especially assessment resources, that can support and inform those strategies. Also, as the number of educational “apps” and “digital resources” scales, the need for effective curating that combines easy access/integration with effectiveness becomes paramount. 
• Digital resource recommendations: Ability to help automate the recommendation of educational resources from among the curated choices will provide high value to instructors, students and parents. This is a very advanced topic that will take many years to develop.  However, it is completely feasible that machine learning techniques could be applied to monitoring and capturing the recommendations of instructional experts.
• Understanding the usage of digital resources: As the use of digital materials scales there are obvious high value questions that need to be answered concerning the usage of digital resources.  Which ones are being used, when and why? Which ones are not being used?  Why not? This information is of great value to suppliers and institutions alike. However, if it is not “easy” to get the data in one place and understand it, the adoption of digital resources will be hampered.
• Optimizing and/or personalizing student pathways to a credential or degree:  The ability to understand student pathway options and choices, improve them and recommend them has obvious value. True personalized learning, an umbrella term that can encompass many things, is achieved when the outcomes are defined for each student. The true personalization occurs then across a set of courses or competencies.  Educational success and fulfillment are achieved through a balance of personalization and optimization of the path through courses and gaining of competencies.
• Better understanding of student progress and risks:  For an NGDLE to deliver on student success it will need to do a better job of helping students understand their level of success and risks of failure.  The NGDLE must support real-time communication and dashboards across a variety of activities to help students measure themselves. An NGDLE will help students, advisors and faculty understand when self-service needs to be supplemented with high touch intervention.

We are observing emerging product categories and how they are fitting into the edtech ecosystem all the time in IMS Global.  I am going to refrain from mentioning specific product names here.  Rather, I will use abstracted definitions of the categories we see growing and emerging.  A very important note, that while the learning management system category is not listed, the LMS has the potential to enable or implement many of the categories below.  Whether or not it is a good business strategy to do so is another question.

• Modular digital curriculum and assessment: Instructional material or curriculum as a “black box” in which the modular construction and progression are difficult to access and understand are becoming less attractive in the digital age. There is both huge opportunity and risk here for organizations that can structure their curriculum products to help solve the high value use cases above. 
• Integrated adaptive assessment: While related to the ability to modularize assessment (previous product category) the category of assessment tools that can be readily integrated with a variety of instructional resources and approaches will be fundamental to improving digital learning outcomes. Assessment is a specialized science and quality assessment is not simply providing some test items. This category will combine ease of use/integration with high quality assessment with real-time feedback.
• Digital curriculum mapping and management: Differentiated instruction, individualized instruction and personalized learning all require management of the pathways and options available to teachers and students. The management of the curriculum needs to take a step up from where it is today. This product category will enable that.
• Learning resource catalogs, app stores, repositories: Perhaps Google or Amazon will be the answer to searching for educational resources. However, the search and integration of resources required to enable the high value use cases above is a lot more complicated than the sort of searching and recommendations that suffice for enabling Internet commerce.  This product category addresses the organizational and ecosystem integration features needed to enable better learning outcomes and ease of use.
• Learning analytics processing and messaging: Learning analytics is a hot new product category. It is also something that many LMS platforms and tools offer in some shape or form.  As with assessment, learning analytics is a highly specialized field. And, analytics needs to be performed across many apps.  Therefore, I think it is likely that new products will emerge that are separate from existing products.
• Outcomes and achievements management: Several new product types are emerging that are helping institutions manage how they structure and assess student progress in terms of competencies and micro-credentials.  Some of these products are incorporating the “integrated” assessment category.  This category will help institutions structure the relationships among the outcomes they hope to achieve with students. It is different but potentially related to curriculum management. It is a layer above.
• Student pathway management: Products that can help advisors and students understand the path.  These products will help students plan and optimize their path to their learning and degree objectives.
• Edtech research data collection and processing: An NGDLE will enable better research of the effect of technology and instructional approaches on student achievement and outcomes. Therefore, I anticipate that products that enable learning analytics correlation of edtech usage to student success will emerge.  This could be considered a subcategory of the “learning analytics” category above, but am calling it out separately as a vey important and necessary contribution that may not be covered by other analytics applications.

Want to read about specific products along these lines? The best place is the IMS Learning Impact Report, which we publish annually, and which we consider to be ongoing research into the product categories that are leading the way to next generation digital learning.

Finally I will leave you with a simple idea and figure (shown here) to help illustrate the road we are on in terms of a maturity model of edtech products and the evolution toward next generation digital learning environments.  At the base level, an educational institution should be able to support the use of technology to support teaching and learning. This is stuff like BYOD and Google Apps for Education or Microsoft Office 365. The next level up is the enhancing of productivity using technology. Herein lies the success of LMSs and many other tools and technologies. However, one should not think it trivial to get to this 2^nd/middle level because there are loads of examples of digital tools, and lack of integration of those tools, that have made life less productive for teachers and students. The “top” tier are emerging applications that have the power to improve student achievement and learning outcomes. Some of these are new product categories and others are existing products that are used in ways that clearly support better outcomes.  But the key point is that for an NGDLE, the expectations are evolving from the perspective of the customers – the students, the faculty, the institutions and society in general. 

The motivation for collaborating on NGDLEs is to enable a more effective ecosystem of educational platforms, apps, tools and resources that have measureable impact.  I hope to see you at the Learning Impact Leadership Institute where a unique collection of sector leaders gathers to further this important collaboration.