Benchmarks of Success for Maryland’s Workforce System

Data and Dashboard Committee

10:00 – 12:00 PM, April 2, 2018

Anne Arundel County Career Center

Attendees: LiLi Taylor, Milena Kornyl, Richard Reinhardt, Sara Johnson, Andre James, Elizabeth Tatum, Ellen Beattie, Joe Raymond, John Stem, Lisa Nicoli, Lynda Weber, Michelle Day, Patricia Morfe, Rob Pinkard, Scott Dennis, Terry Gilleland, and Natalie Clements

Handouts:

Agenda,
Revised Committee Charter,
Revised Global Work Plan,
Draft Glossary of Terms, and
Benchmark Data Definitions

Minutes

I. Opening

Welcome to new DLLR DWDAL Deputy Assistant Secretary Richard Reinhardt.
The next Data and Dashboard meeting will be 10 – 12 pm, Monday, May 7, 2018.

II. News and Notes

DLLR submitted the Maryland Combined Workforce Plan through the federal portal on Friday, March 29, 2018.
The Executive Steering Committee approved the revised Global Work Plan on Wednesday, March 14, 2018. At the meeting, Secretary Padilla told Secretary Schulz to expect updated on MD THINK in a month.
Joe Raymond profiled Maryland’s work on Temporary Assistance for Needy Families (TANF) and the Workforce Innovation and Opportunity Act (WIOA) collaboration in a national webinar. A summary of his presentation will be shared with the committee.

III. Charter and Global Work Plan

The revised charter reflects the committee’s suggestions as well as changes to the Global Work Plan. The goals are organized as “near-term” and “intermediate/long-term”. The committee approves the charter.
The Global Work Plan: The WIOA Alignment Group meets at the end of each month, and the Executive Steering Committee meets quarterly. Committee deliverables will follow that path for approval.
- Quarter 1: Deliverables include the glossary of terms and asset map. These items should be “finalized” (with the knowledge that they are living documents) this quarter. A test Google Doc will be sent to ensure that every participant is able to access and edit it.
- Quarter 2: The Partner Data Collection Plan is a plan of how to go about bringing all of the resources together and what to do with the data.
  - How can this committee balance effort with outcome?
  - How much data is to be shared between agencies?
  - If MD THINK is the solution, then what can be done before it is ready?
- Quarter 3: The committee will begin work on the Data Dashboard and annual activity summary. If needed, the committee can present legislative action to the Maryland General Assembly to free new resources.
It is unknown where the Data Dashboard will be hosted, but ideally all of the state agency websites will link to it. There is a value in posting the finalized glossary and asset map with the dashboard for greater transparency and explanation of the data. Data explanation should be at no more than an eighth grade reading level. The asset map will help the committee address legislative action, if needed.
The program and fiscal years need to be reviewed and resolved on the Global Work Plan.

IV. Benchmarks of Success – Glossary of Terms

The Benchmark Data Definitions represent a preliminary process of identifying numerators and denominators.
How the terms are defined affect how they will be measured (e.g. barriers to employment in terms of WIOA “Individuals with Barriers to Employment” does not account for other external barriers, i.e. transportation, childcare, etc.). How can the committee ensure that TANF participants are included in the measures?
The glossary definitions differ depending on the audience. A lay-person, easy-to-read glossary should be developed for the general public, and a data dictionary will be created for data analysts as explanation of how to collect the data. Does a third executive level explanation of data need to be created?
Milena Kornyl projected the draft glossary and typed updates and notes throughout the meeting. Please see the updated draft glossary for the full list of changes.
The Strategic Goals are overarching themes. Each goal has Benchmarks (measurable outcome statements). Each benchmark contains a baseline (x%) and target (y%). The baselines represent current performance, and the targets are the desired performance. Time is a large factor in determining success. See the top of the next page for a visual representation of these terms.
It is recommended that, for now, the committee include another column on the glossary Excel sheet indicating the source of the definition, so the committee can better keep track and compare sources.
Many of the definitions can be pulled from the Maryland Combined Workforce Plan (e.g. agency descriptions, Maryland Workforce System Customer, etc.). Natalie Clements will pull these definitions.
“Financial Capability” have replaced the term “Financial Literacy” in the financial empowerment world. However, for the sake of this glossary, the term will remain “Financial Literacy” to stay consistent with the Benchmarks, the Maryland Combined Workforce Plan and WIOA.

Charter and Global Work Plan

If terms are not included in the Benchmarks document, do they need to be defined? Terms that are expected to be used to explain the data should be defined. Definitions should consider why each term is included.
- For example, in Maryland, there is no high school diploma (HSD) equivalency. The GED exam results in the award of a HSD. Should “equivalency” be included in the definition of a HSD? This depends on the purpose of including the term. If HSD is included as a measurable skill gain, then there is no need to include “equivalency”. However, if it is used as a starting point, then the committee must consider those individuals that come to Maryland after receiving a HSD equivalency in another state.
The committee has elected to add the following terms, at least, to the glossary: “reportable individual”, “exiter”, and “eliminate” (i.e. eliminate barriers).
Common WIOA definitions can be used in this glossary up to the point that they are consistent with the Benchmarks.
The Benchmarks are not about collecting the data for punitive reasons. They are about defining success for Maryland and determining what direction the state is currently going in. The approach and focus can change; therefore, there is a possibility to include more macro-issues in the future, as it relates to workforce development and outcomes.

V. Benchmarks of Success – Data Asset Map

The committee did not have time to discuss this agenda item.

VI. Next Steps

Volunteers from the committee, representing different WIOA titles, will review and provide definitions for the glossary as relevant to their agencies. Definitions are due back to Natalie Clements by Friday, April 13, 2018. These individuals, and any other interested committee members, will participate on a conference call on Monday, April 16, 2018 to finalize the definitions. Natalie will send a calendar invite for the conference call. Volunteers are as follows:
- WIOA Title I: Patricia Morfe and Milena Kornyl;
- WIOA Title II: Ellen Beattie;
- WIOA Title III: Lynda Weber and Andre James;
- WIOA Title IV: Scott Dennis and John Stem;
- TANF: Lisa Nicoli and Elizabeth Tatum; and
- Maryland Combined State Workforce Plan: Natalie Clements.
The standing committee meeting for June 4, 2018 will be rescheduled for the following Monday, June 11, at the same time (10:00 – 12:00 PM) because the original date conflicts with the full-day professional development event, Next Level Innovation Series.

Benchmarks of Success for Maryland’s Workforce System – Download [Optimized PDF]

Benchmarks of Success for Maryland’s Workforce System – Download