Blackboard

 

An Introduction from Jay Robertson

I hope you are having a great start to 2009.  While the Managed Hosting team works through a busy upgrade period, we’re continuing to focus on your success.  More specifically, we’re hard at work analyzing our client survey results to best determine how your customer experience can be improved.

As Michael Chasen mentioned in his first annual client experience letter, we are also making a number of Managed Hosting investments and client support improvements.

First, in order to better anticipate necessary changes to the datacenter network, we are implementing several industry-leading network management tools.  These tools will help ensure that our network devices have the capacity to handle a growing environment. We believe you will see results in the form of better performance and stability. 

We’re also making important changes to our support model by partnering with Sutherland Global Services for managed hosting client support.  This change is already underway and described in detail in the “Changes within Managed Hosting” section below. We’re continuing to expand our team as well, with new folks in a variety of roles joining us in Europe, Asia, and the U.S.  The “Spotlight” section below highlights our growing DBA team.

Finally, we’ve created additional resource centers containing general Managed Hosting information and up-to-date outage information.  You’ll find details on our new wiki page and information portal in the “Announcement/News” section below. 

We are excited about these projects and hope that you’ll feel their impact in the coming months.  As always, please don’t hesitate to send us your comments and feedback.
Back to Top

 

New Online Tools and Resources

In an effort to provide you with more detailed information about Managed Hosting in general and your Blackboard environment in particular, we are introducing two new tools - the new Managed Hosting Wiki page and the Managed Hosting Information Portal.

The Managed Hosting Wiki Page is an excellent resource to find anything and everything you want to know about Managed Hosting.  Need to escalate a case but don’t know how?  Check out the “Support Escalation Path” under the “General Docs” tab.  Want to learn more about Complex Hosting?  Read the Service Briefs under the “Complex Hosting” tab.  Confused about storage and backups?  Download “The Value of Managed Hosting Storage” from the “Storage Related Documentation” tab.  You can peruse the site using your Behind the BlackboardTM login.

The second tool, which also requires your BtBB login, is the Managed Hosting Information Portal.  The purpose of this portal is two-fold: to provide a calendar of scheduled maintenance and relay information during widespread, unscheduled outages.  The site is located on a server outside of the Blackboard Managed Hosting network so it can still be accessed even if the network is down.  You’ll find information on major datacenter outages with periodic updates and estimated resolution times.

During outages, we know that your Technical Support Managers (TSM) need to work on resolution issues and take calls from other affected clients. As a result, they can be difficult to reach by phone. That’s why this website allows you to get the information you need without waiting on hold or even having to pick up the phone.  Of course, it does not substitute for your TSM. You are still welcome to call if you have additional questions or concerns at anytime.

Note that this portal will not include outages that affect one or even a handful of clients.  This is specifically for infrastructure-related outages that are widespread and affect an entire datacenter.  You will also continue to receive e-mail updates because this website is intended to augment, not substitute, those notifications.
Back to Top

 

Client Support Update

As Jay mentioned in his Support Update letter last fall, it has become increasingly difficult to recruit quality Level I technical support staff in the Washington, DC area.  As a result, we have been exploring other avenues for implementing the best processes, technology, and innovation in the support field. 

The most significant change resulting from this effort is our partnership with Sutherland Global Services.  We’ve decided to move our first line support to Sutherland, largely because of the success that the Local Support team has seen with this model.  Almost a year into the Sutherland implementation, this team is reaping significant benefits with better responsiveness on issues and overall positive feedback from clients.

You may remember that the last newsletter mentioned how we’re moving the Managed Hosting support model to a team-oriented approach where clients would be assigned a team of TSMs instead of a single contact.  This concept will be carried over to the new model with teams of Sutherland TSMs supporting each client.  As a result, improved response times and better case management are the only changes that should be obvious to you. 

This new team of Level I TSMs will still be responsible for handling product issues, scheduling upgrades, requesting backups and all the other tasks that TSMs normally manage.  Any issues they cannot handle will be escalated to a Level II Support team consisting of Client Support Engineers (CSEs) with very specific expertise.  The CSEs will continue to work in our Chantilly, VA datacenter alongside our Operations and Infrastructure teams.

It’s important to note that this new support model does not affect our Complex Hosting clients. They will continue to receive support from their dedicated Complex Hosting Manager (CHM).
 
We are currently in the process of moving our North American clients to this new support model.  We expect to move this subset of clients to Sutherland by mid-‘09.

Thus far it has been a smooth transition, and we appreciate your patience as you go through the process of getting familiar with your new TSM team.  If you have any questions or comments, feel free to e-mail our support management team at supportmanagers@blackboard.com. 
Back to Top

 

Spotlight on the DBA Team

The Blackboard Managed Hosting team is comprised of several groups working together to support you and over 600 other customers.  One of the teams most crucial to the successful maintenance of the Blackboard environment is our team of Database Administrators, or DBAs.  A dedicated team of engineers, the DBAs provide quality database administration services including installations, upgrades, backups, restores, recoveries, migrations and performance tuning.  The team is available 24x7 and is also responsible for troubleshooting and resolving database problems.

The Managed Hosting DBA team consists of 10 database administrators with broad background and experience in database administration, product support, product development and engineering.  70 percent of the team has advanced engineering degrees while 60 percent have more than ten years of experience working with database systems. 

Members of the DBA team are spread across time zones.  Five work in our VA3 datacenter in Chantilly, Virginia, three in Chennai, India, and one each in Vancouver, Canada and Amsterdam, Netherlands.  Dr. Tieng Yap manages the team after spending five years as a database architect in the Blackboard Product Development department.  In his tenure, Dr. Yap has introduced a number of initiatives designed to significantly improve the quality of our Managed Hosting database services, including:

• Detailed deployment plans
• Step-by-step standard operating procedures
• Root cause analyses
• Tool development for automating routine tasks
• Fast backup and restore methods using filer snapshot technology
• Oracle alert log monitoring
• Automatic problem detection and resolution
• An on-call policy and escalation procedure for handling problems outside of regular business hours

The DBA team, with support from our Operations and Monitoring teams, currently manages Oracle databases running releases 9.2 and 10.2 and specializes in Real Application Cluster (RAC) configurations.  The largest database is around 2TB.

Some “quick facts” about the DBAs:

Tieng K. Yap, PhD
Tieng is the Manager of the Managed Hosting DBA team.  He has been with Blackboard for 10 years and with the Managed Hosting team for four years.  He has 11 years of additional experience.  In addition to having numerous industry certifications, including Oracle Certified Professional DBA and Microsoft Certified DBA, Tieng has received several awards for his exemplary work as a member of the Blackboard team.

Hobbies and/or Fun Fact: “I taught graduate courses at Johns Hopkins University for 5 semesters, authored and co-authored one book, two book chapters, four engineering journal articles, and eight peer-reviewed conference papers.”

Jeff Boulier
Jeff has been with Blackboard for seven years, all as part of the Managed Hosting team. He began as a Technical Support Manager, spent several years as an Operations Engineer and joined the DBA team in 2005.  He has six years of Systems Administration experience, all in higher education, and is an Oracle Certified Associate.

Hobbies and/or Fun Fact: “I volunteer as an emergency medical technician with the Nokesville Volunteer Fire Department & Rescue Squad.”

Raymond Konishi
Ray has been with the Managed Hosting team for three years.  Prior to that, he spent four years as a WebCT DBA and consultant providing training and consulting services.  He has a total of 20 years of software development and IT experience, twelve as a DBA.

Hobbies and/or Fun Fact: “I love to create things regardless of the canvas… costumes, software, plastics, electronics, and computers.”

Pon Venkat
Pon has been with Blackboard for four years, all as part of the Managed Hosting team.  He has 19 years of prior DBA experience.  Pon is an Oracle Certified Professional DBA.

Hobbies and/or Fun Fact: “I am a marathon runner and enjoy yoga, tennis, and swimming.”

Tim Käck
Tim has been with Blackboard for almost four years and the Managed Hosting team for six months.  Prior to joining the DBA team, Tim spent two years as a Blackboard Technical Support Manager and another two as a Client Support Engineer.  He works in our Amsterdam office.

Hobbies and/or Fun Fact:  “I travel quite a bit, am interested in operating system kernels, mainly microkernels/exokernels. I’m also involved in some open source projects and play Didgeridoo.”

Sreedhar Chidambaram
Sreedhar joined Blackboard a year ago and has 12 years of prior DBA experience.  He is an Oracle Certified Professional DBA, as well as a RedHat Certified Technician and a Sun Certified System Administrator.

Hobbies and/or Fun Fact:  “Working with computers whether at home or office! That’s fun for me. Otherwise, like to play any kind of games, such as tennis, basketball, pool table, bowling, etc.”

Sannala Srinivas
Sannala just joined the Managed Hosting team in January of 2009, but has 12 years of prior DBA experience.  He has one patent under his belt, received Intel's Recognition Award, and presented at Oracle Open World and National Capitol Oracle User Group on new Oracle security and compliance solutions.

Hobbies and/or Fun Fact: “Chess, table tennis and movies, watching Seinfeld for the past decade without getting bored.”

Sahul Hameed
Sahul has been with Blackboard for just under a year and has almost four years of DBA experience. He is also an Oracle Certified Professional DBA.

Hobbies and/or Fun Fact: “I like to play chess and table tennis.”

Varadharajan C N
Varadharajan has been with the Managed Hosting team for a little less than a year and has four years DBA experience.

Hobbies and/or Fun Fact: “I volunteer in the TamilNadu Science Forum, an organization for popularization of science. I also play cricket and chess, and read books.”

P.C. Harish Chandra
P.C. has been with the Blackboard DBA team for less than 6 months, but has almost five years of experience as a DBA and is an Oracle Certified Professional DBA.

Hobbies and/or Fun Fact:  “Play PC Games, listen to music and love to travel.”
Back to Top

 

Technology Highlight

In each newsletter, we will introduce one or more new technologies that have been implemented in our datacenter in order to improve our services.

Datacenter Changes 

The Managed Hosting Infrastructure team has spent more than a year designing and building a new state-of-the-art datacenter in Chantilly, Virginia. 

The new datacenter, dubbed VA3, is a 7,500 square-foot facility with one Megawatt of power.  This past October, we began installing new client environments into VA3, and transferring existing clients from VA1 to VA3.  All current VA1 clients will be moved to VA3 and as a result, will upgrade to the latest generation of Dell hardware technology.  This process is expected to be completed by June of this year.

We’re also increasing our overall footprint by expanding our VA2 datacenter in the United States and opening a second facility in Amsterdam. With the VA2 expansion, we’re assuring existing clients that they’ll have room to grow in their current space.  Meanwhile, opening the second datacenter in Amsterdam means additional space for new clients, and enables us to provide daily backups to a European datacenter (welcome news to our European client base!). 

Dedicated Fiber Ring

In the fall of 2008, Blackboard connected our domestic datacenters with a dedicated fiber ring in Virginia.  The ring runs at 10Gbps speeds and enables us to connect to Equinix and its numerous transit providers. 
Back to Top

 

Tour a Datacenter

BbWorld® 2009 is taking place in Washington, DC and we’re hoping you’ll take the opportunity to join us for this event and for a tour of our new Chantilly, Virginia datacenter.  It’s a great first-hand opportunity to see the magnitude of the back-up generators and fuel tanks, the many rows of servers and storage devices and the technologically advanced Network Operations area.

If you haven’t been to a datacenter or just want to see where your servers actually reside, this is a great chance to do so. And since our datacenter approach is consistent regardless of location, you’ll get a strong idea of your Blackboard environment even if your servers are not located in VA3.  If you would be interested in taking part in a tour, please contact your Client Manager to schedule either group or individual tours.

If you are not attending BbWorld but find yourselves near Washington, Amsterdam, or Sydney and would like to take a datacenter tour, please let your CM know.  We would be more than happy to accommodate your request.
Back to Top