Wednesday 4 July 2012

SQL Server CPU running at 100% during a full or incremental crawl


We were researching which SQL versions were ok while troubleshooting a problem we had with the SQL Server CPU running at 100% during a full or incremental crawl.
After verifying there were no bottlenecks elsewhere in the farm we turned to SQL and CU version.
Despite there being no errors on the SQL server the SQL guy noticed that some of the SQL queries were not being handled well. 
We were running SQL 2008 R2 Service Pack 1 with Cumulative Update 3 so re-check for recommended SQL versions.


SharePoint 2010 and SQL Hotfix Dependencies
SharePoint Joel has a great article on SQL versions and hotfix dependencies

The above article showed that we should be ok but a short Google later turned up these chaps with the same issue with SQL CPU running at 100%.


An upgrade to SQL 2008 R2 SP1 CU 7 fixed the problem and we were able to do a Search Content source crawl without crippling our SQL server. 



This SQL article has more info on the issue.
Article ID: 2658214 - Last Review: April 12, 2012 - Revision: 5.0
FIX: Poor performance when you run a query that contains correlated AND predicates in SQL Server 2008 or in SQL Server 2008 R2 or in SQL Server 2012
To resolve this issue, apply the following cumulative update. After you apply the update, you must enable trace flag 4137 to add the new logic for the cardinality estimation of AND predicates.
After patching it is necessary to add the switch /4137 (or -4137) as a startup parameter for the patched SQL instance in order to activate the remedy for this particular problem

Monday 2 July 2012

Uninstalling SharePoint Diagnostic Studio 2010

How to uninstall SharePoint Diagnostic Studio 2010

SharePoint Diagnostic Studio 2010 is actually part of the SharePoint 2010 Admin Tookit.


To uninstall a specific tool from the SharePoint 2010 Administration Toolkit
1.  From Programs and Features, select the Microsoft SharePoint 2010 Administration Toolkit and click Change.
The same dialog box is displayed if you run the SharePoint2010AdministrationToolkit.exe file instead of using Programs and Features.
2. Select the tool you want to remove.
3. Click Next, and then click Finish.

Detailed here : http://technet.microsoft.com/en-us/library/cc508848


Note : installing the tool provisions a solution globally "extendeddiagnosticproviders.wsp", this is not removed when you un-install the tool.



Also note, SharePoint Diagnostic Studio (SPDiag) 2010 can place significant load on SharePoint servers
http://support.microsoft.com/kb/2655725

MS say :
Using the SharePoint Diagnostic Studio (SPDiag) 2010 can consume significant network bandwidth and other resources on the server being monitored as well as on the database server used to host the project database.
Evaluate the need to have the toolkit installed.
Uninstall the SharePoint Diagnostic Studio 2010 if it is not required.


It's nice to have clean working performing environments.

Monday 18 June 2012

Essential tools

Some great everyday tools to work with. These are mostly based around work with SharePoint.


SharePoint related

ULS viewer - http://ulsviewer.codeplex.com/ or http://archive.msdn.microsoft.com/ULSViewer/
don't leave home without it. ULSViewer allows users to open a ULS log file and display its contents in a user friendly format. Users can then perform advanced functions such as filtering, sorting, highlighting, loading logs, appending logs, etc in order to single out the data that is important to the user. This information can be used to diagnose problems with machines running ULS services, or to monitor machines and the events they create in realtime.


Notepad++ - http://notepad-plus-plus.org/
fantastic tool for html, xml, config file editing

XML Notepad -  http://xmlnotepad.codeplex.com/
XML Notepad provides a simple intuitive User Interface for browsing and editing XML documents

WinMerge - http://winmerge.org/
compares files, go compare

Power GUI - http://powergui.org
GUI for script editor for Microsoft Windows PowerShell, more features and can be more stable in some cases than PS console

Fiddler - http://fiddler2.com/fiddler2/
Browser Web Debugging Proxy, logs traffic between PC and browser.

WireShark - http://www.wireshark.org/
 go deep, network protocol analyzer for Unix and Windows




Useful Firefox\IE add-ins

Firebug - http://getfirebug.com/
Tools for web development. Allows inspect, edit and monitor CSS, HTML, JavaScript and Net requests in any web page

Yslow - http://yslow.org

Grade your web pages
Page performance, page components, stats, tools for performance

HttpWatch - http://www.httpwatch.com/
HttpWatch is an integrated HTTP sniffer for IE and Firefox that provides new insights into how your website loads and performs



Day to day tools

Vision app remote desktop - http://www.visionapp.com
excellent tool to manage remote access to multiple systems all at once, set your creds and login with one click. copy and paste between systems.


TreeSize pro - http://www.jam-software.com/treesize/
GUI for local and remote hard disk manager

KeePass - http://keepass.info/
Local client password manager

VMWare Workstation
VMWare machines at will


Process Explorer - http://technet.microsoft.com/en-us/sysinternals/bb896653.aspx

Process Explorer shows you information about which handles and DLLs processes have opened or loaded.

Sunday 29 April 2012

Sharepoint webpages slow to load (SPCertificateValidator.Validate) long execution time

Has been a week of make, break, fix and make it faster.
Performance issues with SharePoint can be caused by many issues. This is one small avoidable issue that can help improve the page load speeds.

As a general rule the first thing to start with when troubleshooting performance issues with web page loads times, is to enable the developer dashboard. This will give you a break down of the components the page loads and the associated load times.
To enable the Dev Dash : http://workingsharepoint.blogspot.co.uk/2012/03/developer-dashboard.html

One item we found consistently coming up with a long load time (20s) during testing was the following. (SPCertificateValidator.Validate). Execution Time=20034.2566316328

A quick explanation of this is that a component on the page is trying to check the validity (CRL check) of the "SharePoint Root authority" certificate. As it cant find it in the Trusted root certificates, Windows tries to retrieve the CRL from the net and causes the delay.
See the MS article for a full explanation.



Solutions to this are to do one, or both if a test environment, on your WFE servers.
This comes straight from the MS article which is referenced below.

A> Install the SharePoint Root Authority certificate in the Trusted Root Certification store.

1. Obtain the “SharePoint Root Authority” certificate as a physical (.cer) file
   a. Launch the SharePoint 2010 PowerShell window as Administrator
   b. $rootCert = (Get-SPCertificateAuthority).RootCertificate
   c. $rootCert.Export("Cert") | Set-Content C:\SharePointRootAuthority.cer -Encoding byte

2. Import the “SharePoint Root Authority” certificate to the Trusted Root Certification store
   a. Start | Run | MMC | Enter
   b. File | Add/Remove Snap-in
   c. Certificates | Add | Computer account | Next | Local computer | Finish | OK
   d. Expand Certificates (Local Computer), expand Trusted Root Certification Authorities
   e. Right-click Certificates > All tasks > Import
   f. Next | Browse | navigate to and select C:\SharePointRootAuthority.cer | Open | Next | Next | Finish | OK



B> Disable the automatic update of root certificates on the SharePoint Servers.

1. Launch gpedit.msc as a local administrator.
2. Go to Computer Configuration | Windows | Security settings | Public Key Policies | Certificate Path validation settings.
3. On the Network Retrieval tab, define the policy and uncheck “Automatically update certificates in the Microsoft Root Certificate Program”
4. Run gpupdate /force for the policy to take effect immediately.


In our development environment we enabled both as there is a minimal risk but is not reccomended for any public facing Production Environment.
The article was for a search page but in our case was happening on the Home page of our site so this helped increase the page loads time by around 20 seconds.


MS Article : Search results are returned much slower intermittently
http://support.microsoft.com/kb/2639348

Monday 23 April 2012

Multiple login prompts with Load balanced Web Front Ends using ADFS Authentication

Problem :
Users were getting multiple authentication prompts when attempting to access a SharePoint website using ADFS authentication.



Possible other issues :

You may randomly be redirected back to a login page.
You may end up in an authentication loop that causes ADFS to halt the request because of a perceived denial of service (DOS) attack, as the note states.
If you look at a trace of the activity, you may see SharePoint setting your fedauth cookie to an expired value, then start making the requests again to ADFS, which then, either won’t issue you a non-expired cookie, or SharePoint looks at and transforms it to an expired cookie.



A quick Google, came up with the site below. (Cheers Steve)

As the our new Network Load Balancer (NLB) was not managed by us, they had not setup cookie based persistence. Or “Affinity” as described in the article below.

This was enabled and the problem went away.

It happens as the traffic from your browser may hit one WFE, and then another. A token is issued for each session so in this case setting cookie based persistence resolves the issue.



Resolution :
Enable the cookie based persistence on the NLB





Steve Peschka
Make Sure You Know This About SharePoint 2010 Claims Authentication - Sticky Sessions Are REQUIRED
http://blogs.technet.com/b/speschka/archive/2011/10/28/make-sure-you-know-this-about-sharepoint-2010-claims-authentication-sticky-sessions-are-required.aspx

Wednesday 18 April 2012

Move SharePoint content to another Farm by backing up and restoring the SQL DB


At some point you will need to restore/move/backfill content from a Production Farm to a test or developer Farm.
You can do this by copying the SQL DB to your test SQL server and reattaching the DB in CA to the Target web application.

A few things to remember :
You will have to have the same solutions deployed on your target Farm.
Check the dependencies of any service applications i.e. Managed Metadata Service & others...
Make sure you have the same SQL server versions on target and source.

These instructions are for SQL Server 2008 R2 and SP 2010.
It is also possible to do this via PS, check the article at the end for further details.


On SQL Server
Get the current SQL permissions set on your target SQL DB.

Check the permissions set on the DB.

1.      SQL DB Properties > Files > Owner > record the account listed
2.      SQL DB Properties > Files > Permissions > record the accounts listed
3.      Security > Logins > accounts from step 2 >  Properties > User Mapping > select the target DB name > select source DB > record Permissions listed

Create a SQL DB backup of your Source web application DB.
Right click your target DB > Tasks > Backup > Backup type > select Full
Under Destination > Click remove > Click add > File name > browse … > Select your backup dir > enter a relevant filename > click ok > ok
Click OK to kick off your backup.
Copy the source SQL DB backup to your Target SQL Server.


To have a recovery option in case this doesn’t work. Backup your target SP web application DB either in SQL or via a SP Farm backup, both if you want to cover all bases.



On your SP Farm
At this point I normally stop the IIS sites on all SP WFE’s that use this target SQL DB to try stop connections to the DB’s. timer jobs may still kick off though.

On your target SP Farm, navigate to > Central Admin > application management > Databases > Manage content databases > Select you web application > select the target DB Name.
Select “Remove content DB” > OK


On SQL Server
On the target SQL server, restore the source DB over the Target DB
Check there are no connections to the source DB.
In SQL mgmt Studio, right click the server > Activity monitor > expand processes > filter by target db > right click and Kill Process on any connections.
Close the activity Monitor pane.

Right click your target DB > Tasks > Restore > Database.
Under To Database > ensure the target DB is selected.
Unser Source for restore select From Device > browse … > backup media File > Click Add > browse to backup location and source DB Backup file > click ok > Ok
Select the backup sets to restore > check the box
Select Options on the left > restore options > Overwrite the existing DB (with replace)
Click OK to kick off the restore.

Once completed, refresh the DB view, right click the restored DB > Files > Change the Owner to the correct account listed in step 1. Above (normally your SP farm account)
Click ok
Expand the target DB > select security > Users > remove any accounts from your source SP farm.

From the top level of SQL mgmt Studio, Select the server > Security > Logins > Select the equivalent account listed above in step 2. for your target SP Farm.
Right click > properties > user mapping > Users mapped to this login > select the target DB name > enable the permissions listed from step 3. above > click OK.

Your target DB should now have the same permissions set as before the restore.


On your SP Farm
On your target SP Farm, navigate to Central Admin > application management > Databases > Manage content databases > Select you web application > click “Add a Content DB”
Ensure the correct Web application and Database Server is listed.
Under “Database Name” Enter the target DB name.
Enter any configuration specifics, if any for the rest of the DB settings.
Click OK.

Your restored DB should now be attached successfully.
Start all IIS sites that use the Target DB’s is you stopped them.
Reset IIS on your WFE’s if you can.  
If you have a warm up script for your target web application, run this then access the site.

The site should eventually come up, if you got all your SQL permissions correct.
It can be really slow sometimes so be patient.


Natalia Tsymbalenko has some common errors in this useful article.
How restore a SharePoint 2010 content database on the different farm :

Tuesday 17 April 2012

Configure BLOB cache settings in SharePoint 2010


Blob cache is a quick and easy way to speed up your SharePoint site.
Here’s the MS blurb
A BLOB cache is a disk-based cache that stores binary large objects (BLOBs) such as frequently used image, audio, and video files, and other files that are used to display Web pages. Each front-end Web server maintains its own BLOB cache. When you enable a BLOB cache, you specify the file types to include in the cache and also the location of the BLOB cache. The first time that a BLOB file is requested, the file is copied from the database to the BLOB cache on the front-end Web server. Future requests to the front-end Web server for that same file are then served from the file that is stored in the BLOB cache, instead of being served from the database. This reduces the network traffic and the load on the database server.


To Enable Blob Cache

1. Make a copy of your web.config, open the live web.config

2.   Find the line <BlobCache location="" path="\.(gif|jpg|jpeg|jpe|jfif|bmp|dib|tif|tiff|ico|png|wdp|hdp|css|js|asf|avi|flv|m4v|mov|mp3|mp4|mpeg|mpg|rm|rmvb|wma|wmv)$" maxSize="10" enabled="false" />

3.  Change the location to a location that exists on each of your WFE’s. (Each WFE will have a cache dir, doesn’t have to be a share)

4.  Amend which files you want to cache by removing those you don’t want to cache.

5. Change the maxsize variable to something your drive location can cater for, size is in GB. (Note each IIS website will use this value, 3 sites = 30GB if set to 10)

6. Change the enabled attribute to “true” from “false”

7. Save the web.config file and load your site.

8. Check the blob cache gets created on each WFE.



To Flush the BLOB cache (SharePoint Server 2010)

When you flush the BLOB cache, you clear the contents of the BLOB cache for a Web application. This is useful if the BLOB cache becomes out of sync with the content. For example, after you restore a content database, the BLOB cache will be out of sync with the content. To correct that situation, you must flush the BLOB cache.

Create a PS script file containing the following.

$webApp = Get-SPWebApplication "<MyWebApplicationURL>"
[Microsoft.SharePoint.Publishing.PublishingCache]::FlushBlobCache($webApp)
Write-Host "Flushed the BLOB cache for:" $webApp

Save the file “FlushMyWebApplicationURL”, and test run it to see that the Blob Cache gets deleted.
Test show this  updates the files in the blobcache dir but doesn’t remove all the files entirely.


 More info on the MS site here :