Ministry of Social Development bosses pushed back against initial ‘poor’ performance score in an evaluation of the $19b Covid wage subsidy
In August last year, government officials and consultants assembled in a meeting room in a central Wellington high-rise building to hear the early findings of a review of the wage subsidy scheme.
Ostensibly, it was a meeting of the wage subsidy evaluation steering group, chaired by the Ministry of Social Development. Other officials from Inland Revenue, Treasury, the Business Ministry, and MSD – from an affiliated “working group” – were also present.
Consultants MartinJenkins presented a summary of its draft process evaluation report, which had its genesis in Auditor General John Ryan’s recommendation, in May 2021, for a timely evaluation of the pandemic-related scheme which, all told, cost $19 billion.
READ MORE: * Verdict on $19b wage subsidy lands, a year late * Top 10 employers asked to repay wage subsidies
Overall, the scheme was found to be good.
It was developed and rolled out quickly, the presentation said, it was easy to access, it paid out quickly, and it was greatly valued. Many workers and employers described it as a lifesaver.
However, three key weaknesses were identified.
Ineffective “programme-level governance” was listed as the third weakness, and more active consideration of “impact on equity” was the second.
The main flaw, according to MartinJenkins, was “the Crown’s delivery of its role as a Treaty partner”.
Minutes from the steering group meeting state: “There were opportunities to take a more Treaty-based approach in the design of the scheme that were not taken, there was little evidence of Treaty-based analysis in the development and design phase, existing mechanisms and relationships were not leveraged, and experience was variable.”
The Crown’s performance on consistency with the Treaty of Waitangi was initially rated as “poor”.
(MartinJenkins’ analysis was done with Te Paetawhiti Ltd.)
By November last year, the first draft of the final report had undergone “significant revisions”, and a final draft, incorporating changes, had received good feedback from the working group and “internal agency consultations”.
It seemed to be sailing towards publication.
The steering group had signed-off the report in principle, and it had been sent to the Ministry of Social Development’s publications committee for checking, with an expected turnaround of two days.
Further feedback was expected from Treasury. Though there had been “some level of socialisation” with agency senior leadership teams, more was expected, especially with the ministry and Treasury.
Social Development Minister Carmel Sepuloni and Finance Minister Grant Robertson were expected to read the report in December, and a public release, alongside several other evaluations, was anticipated in March this year.
The reports were published in July, without fanfare. In the finalised process evaluation report, the wage subsidy scheme’s consistency with the Treaty of Waitangi was rated “fair”, not poor.
(Fair was defined as: few core components met, weaknesses on several components – showing poor prioritisation of effort. Poor, meanwhile, meant there was “clear evidence of unsatisfactory performance/results”, with no or limited evidence of components being met, and “serious weaknesses” with “very poor” prioritisation of effort.)
A joint Ministry of Social Development-Treasury paper to Sepuloni and Robertson, written in July, noted the MartinJenkins report rated the scheme’s performance as very good, good or fair in “all six key areas of development and delivery”.
Consistency with Te Tiriti was “found to be a balancing act in a crisis situation”.
Steering group minutes, released to Newsroom under the Official Information Act, help explain what happened behind the scenes.
Further data collection
The MartinJenkins report had been “effectively signed out” last year, and went through the Ministry of Social Development’s publications committee, steering group minutes from an online meeting in April said.
But “further data collection” led to minor changes – “mainly that the Te Tiriti finding in the evaluation rubric shifted from poor to fair”.
Work was underway to get “some more transparency” about what was described as the second round of data collection “that happened after initial findings were presented”.
A month later, the steering group’s minutes said the ministry’s leadership team raised concerns about the Treaty finding being “inaccurate”, complaining they “and other relevant people” hadn’t been spoken to.
The idea of senior ministry officials – who were already aware of the findings of the report, which had been commissioned by their agency – being interviewed, “could raise methodological issues”, the minutes said.
Once the interviews were done, and the finding changed from “poor” to “fair”, there were calls from officials on the working group for the situation to be detailed, transparently, in the report.
The Ministry of Social Development provided MartinJenkins with an extra sentence for the report, “to provide more clarity” but, after further discussions, “it was agreed that the new wording would be set aside”.
Here’s what the final MartinJenkins report said, on page 41 of 137: “Additional information was provided late in the evaluation that changed our assessment in some key areas. This specifically related to the Crown’s efforts to engage with Māori. In the time provided, we took the information that was provided on face value and were not able to confirm the impact of the engagements that did occur with the iwi leaders who were party to them.”
The report didn’t state where the additional information came from or which assessment had been changed.
We asked the ministry who was interviewed and when, and whether interviews were conducted with individuals who were already aware of the MartinJenkins report’s findings.
Ministry of Social Development’s group general manager of insights Fleur McLaren says: “The evaluation was carried out independently of government agencies.”
In answers to further questions, McLaren says the ministry’s leadership team wanted to ensure key senior officials involved in administering the scheme contributed to the evaluation.
That was the original intention of the evaluation, she says, but it didn’t occur partly due to changes in key personnel at the ministry. “MartinJenkins then completed those additional interviews as originally planned.”
MartinJenkins partner Sarah Baddeley confirms a small number of further interviews were done. She doesn’t name the interviewees, either.
“We were provided information about engagement that occurred with iwi leaders within wider ministerial-led Covid-19 hui,” Baddeley says.
“This information led to a slight improvement in one of the evaluation criteria related to consistency with the Treaty of Waitangi.”
According to the report’s executive summary, the process evaluation assessed how well the wage subsidy scheme was developed and delivered. It would be interesting to know the nature of engagement that was sufficient to improve the score.
Baddeley adds: “Changes of this nature can occur particularly when there are changes in staff or when we are evaluating policy processes that occurred under Covid urgency, and information was held across multiple sources.”
Asked if it was ethical for an external consultant’s finding to be changed after pushback from the ministry’s leadership team, McLaren says: “It’s important that the findings accurately represented what the wage subsidy scheme achieved, given the large amount of public money that was spent on the scheme.
MartinJenkins provided its revised draft report in October last year, and, after “additional fieldwork”, the final draft was provided in February.
The final report, made public five months later, said the consultancy wasn’t able to confirm the impacts of engagements with iwi leaders, “in the time provided”.
Could have done better
Dr Barbara Allen, deputy head of Victoria University of Wellington’s School of Government, says a finding of “fair” is still not very good.
“Collaborations and consultations underpinning policy development can be quite limited.
“It looks to me like MSD realises it could have done better on this, and no one seems to be claiming they got it right.
“We know that we are still learning when it comes to honouring the Treaty in all Crown-Māori relations and policy interaction.”
Mike Smith (Ngāpuhi, Ngāti Kahu), of Northland, was co-leader of the National Iwi Chairs’ pandemic response group, which had regular briefings from ministers and senior public servants.
He can’t recall details of what was said about the wage subsidy scheme.
What Smith found particularly distasteful was big, successful companies in effect passing on the wage subsidy’s benefits to shareholders, as dividends, rather than to their workers. (This was a central theme in Auditor-General Ryan’s 2021 report.)
Taxpayers are probably more concerned about the apparent misuse of wage subsidy money than Treaty compliance, Smith says – “and so would I”.
“I’ll say that as a Māori person. I’d be more concerned about that than I would be about any non-compliance over the Treaty issues.”
In brighter news for the ministry, the evaluations were delivered under budget. Cabinet approved $1m for the work in March 2021, and the final cost was $810,000.
