iCIT team restores service after McAfee gaff

When a faulty McAfee update released last Wednesday morning caused a network-wide Windows XP crash, many iCIT employees were called on to manage restoration of computer functionality across campus.   

 “I’d like to thank each and every one of you who stepped up, putting in extra effort, time and hard work, to help manage this situation in a smooth and timely fashion, even when there were many ‘unknowns” to deal with,” said Elena Pokot, CIO.

To recap:  The security firm McAfee provides daily updates to its subscribers.   Wednesday’s update (DAT 5958),  misidentified “svchost.exe” file in Windows XP Service Pack 3 (SP3) as malware.  The false positive detection caused “svcholst.exe” to be quarantined or deleted, locking computers with Windows XP SP3 in  re-boot mode, and disconnecting them from the network.

At UW-W, the iCIT response was quick.  Distribution of the update was disabled within 40 minutes.  The fix provided by McAfee, which required machines to restart in order to restore connectivity, was in hand by 11 a.m.   The iCIT team restored computer labs by 11:15 a.m., and most personal workstations were up and running by 3 p.m.  In some cases, the affected file was deleted, and in those situations, the fix requires a technician to visit and restore the PC.   Those computers are being restored on a case-by-case basis as the Technology Service Center is being notified.

The faulty release impacted businesses and institutions across the globe, including Intel, Dish Network, several major hospitals and other institutions.   For more information, attached are links to ZDNet and Computer World:

http://blogs.zdnet.com/Bott/?p=2003

http://www.computerworld.com/s/article/9175928/The_McAfee_update_mess_explained?source=CTWNLE_nlt_pm_2010-04-22

Personnel Update: Search Underway for TSC Manager

The position for Technology Support Center Manager (formerly Helpdesk Manager) in iCIT has been posted and the search for a manager to fill this role is underway.  External advertising for the position will begin this week.

Since June 2009, the role of Helpdesk Manager has been filled on an interim basis by Crystal Herron, who has done an outstanding job in overseeing the day-to-day Helpdesk operations.  She will continue in this role for the next few weeks.   Crystal  is to be commended for her service.  Please join us as we thank her for her dedication, and wish her and her husband Joel the best as they begin their family.   Thank you, Crystal!

To assure we do not have an interruption of performing responsibilities, Lisa Rowland will provide coverage for the Helpdesk, in addition to her HP SM duties, as needed until the new manager is hired. 

The Technology Support Center Manager will be a key position to the iCIT organization, and campus.  Those who are interested in being a part of the search process should contact Kent Steinike or Elena Pokot.

HP Service Manager Project Update

We are pleased to report that the few mission critical technical issues with HP Service Manager have now been resolved and we are ready to test the production environment.

The Plan:
The server administrators will be transferring data to the production environment this week and I will be finishing the Service Manager training for iCIT through the early part of March (the last training session is scheduled for March 8th). When the training is complete, the helpdesk will enter tickets in both HP Service Manager and MAGIC for a few days to ensure that everything is working correctly (categories, escalations, notifications, etc.).

Providing there are no major issues, ALL tickets will be entered into Service Manager beginning March 15th and entering tickets into Magic will be discontinued. Technicians may continue using Magic through March 30thfor previously entered tickets; however, on March 31st, all tickets in Magic will need to be resolved or transferred into Service Manager.

Modifications Needed:
A service called “Service Manager Modifications” has been created in Service Manager for any modification requests. Requests will be prioritized and tweaks to the system will continue following the go live date.

Other Area Involvement:
Training will be scheduled during the last part of March, early April for the College and Administrative Tech Liaisons who will also be using Service Manager. Once the Tech Liaisons have a good handle of the process and we have worked out any bugs, we will implement the self-service portion of Service Manager to the campus (before the end of the Spring semester).

There will be more information to follow. Please contact Lisa Rowland if you have any thoughts or concerns.

AV Services Move to CEG Team

In order to better align services provided by iCIT to the campus community, we will be merging AV-related classroom technology services with lab and classroom support.  To effect this change, Shelly Allen will be joining the Computing Environment Group (CEG), and reporting to Mary Jo as part of the CEG team.

Organizational Update- Joe Friedel

In the past several months, Joe Friedel has taken on additional responsibilities in workstation management and software support for office, lab and classroom workstations. In order to best align these current responsibilities with iCIT organizational structure, Joe will be transitioning from Customer Technology Services & Support (CTSS) to the Computing Environment Group (CEG).  The transition will allow Joe to participate in the on-going development of the managed workstation environment for campus.  The organizational change will take effect on Monday, December 7.