Scotland’s Deputy First Minister has said it is “frustrating” the controversy around the SNP has overshadowed Humza Yousaf’s first major parliamentary speech, as she urged the party to get its “house in order”.

Shona Robison spoke to journalists as SNP treasurer and MSP Colin Beattie was the second person to be arrested and released without charge “pending further investigation” into the party’s finances.

The arrest came on the same day as the First Minister aimed to set out a “fresh start” as he detailed the Scottish Government’s priorities for the next three years.

He also confirmed delays to the introduction of the National Care Service and the deposit return scheme.

First Minister Humza Yousaf delivered a statement on his Government’s priorities at Holyrood on Tuesday (Jane Barlow/PA)

As Police Scotland investigate how more than £600,000 of SNP donations earmarked for independence campaigning was spent, Ms Robison said she hopes her Government can move on to focusing on issues that matter to the public.

She also said Mr Beattie, who is deputy convener of Holyrood’s Economy and Fair Work Committee and sits on the Public Audit Committee, should determine whether he should remain in the posts.

During a visit to Dear Green Coffee Roasters in Glasgow on Wednesday, she said: “Of course it is frustrating that we are not able to talk as much as we would like about the priorities that were set out in the prospectus yesterday, and I think households and people who are struggling with inflation divs today, who are struggling with the cost of things… will want to know what we as a Government are doing.

“Although the public will expect the SNP to get its own house in order, they also expect us as a Government to help their households with the cost-of-living pressures.”

Asked if Mr Beattie should remain party treasurer, she said: “These matters are matters firstly for Colin Beattie himself to reflect up but also in discussion with the First Minister and others.”

Earlier this month, former party chief executive Peter Murrell was arrested and questioned for more than 11 hours before being released pending further investigation.

Speaking on BBC Radio Scotland’s Good Morning Scotland programme earlier on Wednesday, Ms Robison said decisive action is “absolutely critical” and that going forward the governance of the party needs to be “absolutely about transparency, openness”.

Deputy First Minister Shona Robison visited Dear Green in Glasgow on Wednesday (Andrew Milligan/PA)

She declined to comment on whether former first minister Nicola Sturgeon might be arrested, but said she does not know if the ex-leader has spoken to police.

Ms Robison said: “We need to wait for the facts to be established, whether that’s through the ongoing police investigation and of course the review about how the party is managed, and that of course is really, really important not just to reassure party members, but it is important that the public expect us to get our own house in order, while also focusing as a Government on supporting their households through the cost-of-living crisis…”

Defeated SNP leadership candidate Kate Forbes, meanwhile, told BBC Radio 4 that claims about the party’s finances had been “mind-blowing” and “decisive and quick action” is needed or the SNP will be in trouble.

Ms Robison told BBC Scotland: “On the decisive action, it is absolutely critical, Kate Forbes is right, and that’s why of course Humza Yousaf has taken that decisive action by talking about that need for a fresh approach to governance within the party.”

Scottish Conservative chairman Craig Hoy said Ms Robison is “living on a different planet” to suggest the First Minister has taken “decisive action”.

He said: “Nobody can say with any credibility that Humza Yousaf is unaccountable for what happened in the past. He made it clear he is the continuity candidate and wanted to carry on the work of Nicola Sturgeon.

“The SNP are in complete meltdown and totally distracted from Scotland’s real priorities. Only the Scottish Conservatives are focused on those real priorities, like the cost-of-living crisis, fixing our NHS and strengthening the economy.”

Scottish Liberal Democrat leader Alex Cole-Hamilton said: “We are in the ridiculous situation where the First Minister cannot categorically state that his party isn’t a criminal organisation.

“It’s hard to get your house in order when it’s been blown to smithereens.”