Skip to content
Tech Witch
Primary Navigation Menu
Menu
  • Home
  • Who is TechWitch?

The case of the missing System Writer

By: TechWitch
On: April 5, 2018
In: Windows
Tagged: backup, vss, windows

Recently I was working on a problem for a Windows Server 2008 system where the daily incremental backups were failing. These backups had never worked, but the weekly full backups were successful.

First step was reviewing the failed backup logs (these were CommVault logs), and you could see the following:

commvault log

and

So, the next thing you do is run up a command prompt and check the vss writers on that server:

The system writer was not listed.

Next step was to check if there were any conflicting tasks such as AV scans running at the same time as the backup, there weren’t any.

I then went on to review the system and application event logs to see if there were any VSS errors. Interestingly there were none of these either.

Of course I had to try the reboot option (just to see if it would work), but nope, this didn’t help.

Researching lead me to the usual Microsoft articles on missing vss system writer:

KB 2009272 and a blog article on missing system writer case explained.

Although I didn’t see any VSS errors in the event logs, I decided to double check the permissions for the two directories mentioned in KB 2009272 anyway just to be on the safe side. As I did not have direct access to the end system I tried to be “smart” about this and decided to use PowerShell to check the permissions. With a couple of lines of PowerShell I could get the File Permissions and the Directory Permissions.

File Permissions:

Get-ChildItem “c:\windows\winsxs\filemaps” | Get-Acl | export-csv c:\filemaps_fileperms.csv

Get-ChildItem “C:\Windows\winsxs\Temp\PendingRenames” | Get-Acl | export-csv c:\pendingrenames_fileperms.csv

Directory Permissions:

Get-ChildItem -Path ‘C:\Windows\winsxs’ -directory -Filter “*filemap*” | get-acl | export-csv c:\filemaps_permissions.csv

Get-ChildItem -path ‘C:\Windows\winsxs\Temp’ -directory -filter “*pendingnames*” | Get-Acl | export-csv c:\pendingrenames_permissions.csv

Unfortunately I was being a bit too clever for my own good. Simple screenshots would have been much easier in the long run, as I hadn’t taken into account the way PowerShell formats the output it generates. Ultimately all the permissions were correct, which I kind of suspected given that I didn’t have any of the corresponding VSS errors in the event logs.

Proceeding onwards I reviewed the second Microsoft article and got the Microsoft.Net cache folders to be checked and if any files present to clear them out and restart the cryptographic service. There were not however any temp files in Microsoft.Net cache directories, so another dead end.

I was out of ideas, I had done all the suggestions in the articles (without running Procmon), but it had all been covered off. So I bit the bullet and raised the case with Microsoft.

What did they do? Confirm that the system writer was missing by running vssadmin list writers, checked the event logs for VSS errors (none present) then use Procmon.

  1. Start Procmon logging.
  2. Run command vssadmin list writers at an elevated command prompt.
  3. Stop Procmon logging.

On reviewing the log file they saw the following:

So the setupapi.ev1, setupapi.ev2 and setupapi.ev3 files are locked.

Referring back to the Microsoft blog I mentioned earlier, you can see the registry key entry it mentions.

Next the Microsoft engineer had a look at the 3 locked files in the c:\windows\inf directory and compared them to a system with the same OS version to check file sizes. There was a difference. Microsoft said that these 3 files were corrupt.

To resolve the problem, all we had to do was copy setupapi.ev1, setupapi.ev2 and setupapi.ev3 from the working system (same OS level) to the problem one and then restart the cryptographic service.

Testing with an incremental backup was then successful!

 

 

2018-04-05
Previous Post: First Steps with Wireshark
Next Post: Enabling Project for a user in Office 365

This is my personal blog all of the opinions expressed are mine and mine alone.

Recent Posts

  • Micro:bit Challenge
  • O365 Reports for Teams and User Activity
  • Teams Expiration Policy and Deleting O365 Groups
  • Missing Office Splash Screen
  • Spellcheck in MS Teams

Archives

  • July 2020
  • June 2018
  • April 2018
  • April 2017

Categories

  • 365
  • Debugging
  • Office
  • Programming
  • Python
  • Reporting
  • SCCM
  • Teams
  • Uncategorized
  • Windows
  • Wireshark

Meta

  • Log in
  • Entries feed
  • Comments feed
  • WordPress.org
Caroline Knight
Tweets by CarolineK_IT Website security

Designed using Dispatch. Powered by WordPress.