Sharepoint webpages slow to load (SPCertificateValidator.Validate) long execution time

Has been a week of make, break, fix and make it faster.
Performance issues with SharePoint can be caused by many issues. This is one small avoidable issue that can help improve the page load speeds.

As a general rule the first thing to start with when troubleshooting performance issues with web page loads times, is to enable the developer dashboard. This will give you a break down of the components the page loads and the associated load times.
To enable the Dev Dash : http://workingsharepoint.blogspot.co.uk/2012/03/developer-dashboard.html

One item we found consistently coming up with a long load time (20s) during testing was the following. (SPCertificateValidator.Validate). Execution Time=20034.2566316328

A quick explanation of this is that a component on the page is trying to check the validity (CRL check) of the "SharePoint Root authority" certificate. As it cant find it in the Trusted root certificates, Windows tries to retrieve the CRL from the net and causes the delay.
See the MS article for a full explanation.



Solutions to this are to do one, or both if a test environment, on your WFE servers.
This comes straight from the MS article which is referenced below.

A> Install the SharePoint Root Authority certificate in the Trusted Root Certification store.

1. Obtain the “SharePoint Root Authority” certificate as a physical (.cer) file
   a. Launch the SharePoint 2010 PowerShell window as Administrator
   b. $rootCert = (Get-SPCertificateAuthority).RootCertificate
   c. $rootCert.Export("Cert") | Set-Content C:\SharePointRootAuthority.cer -Encoding byte

2. Import the “SharePoint Root Authority” certificate to the Trusted Root Certification store
   a. Start | Run | MMC | Enter
   b. File | Add/Remove Snap-in
   c. Certificates | Add | Computer account | Next | Local computer | Finish | OK
   d. Expand Certificates (Local Computer), expand Trusted Root Certification Authorities
   e. Right-click Certificates > All tasks > Import
   f. Next | Browse | navigate to and select C:\SharePointRootAuthority.cer | Open | Next | Next | Finish | OK



B> Disable the automatic update of root certificates on the SharePoint Servers.

1. Launch gpedit.msc as a local administrator.
2. Go to Computer Configuration | Windows | Security settings | Public Key Policies | Certificate Path validation settings.
3. On the Network Retrieval tab, define the policy and uncheck “Automatically update certificates in the Microsoft Root Certificate Program”
4. Run gpupdate /force for the policy to take effect immediately.


In our development environment we enabled both as there is a minimal risk but is not reccomended for any public facing Production Environment.
The article was for a search page but in our case was happening on the Home page of our site so this helped increase the page loads time by around 20 seconds.


MS Article : Search results are returned much slower intermittently
http://support.microsoft.com/kb/2639348

Multiple login prompts with Load balanced Web Front Ends using ADFS Authentication

Problem :
Users were getting multiple authentication prompts when attempting to access a SharePoint website using ADFS authentication.



Possible other issues :

You may randomly be redirected back to a login page.
You may end up in an authentication loop that causes ADFS to halt the request because of a perceived denial of service (DOS) attack, as the note states.
If you look at a trace of the activity, you may see SharePoint setting your fedauth cookie to an expired value, then start making the requests again to ADFS, which then, either won’t issue you a non-expired cookie, or SharePoint looks at and transforms it to an expired cookie.



A quick Google, came up with the site below. (Cheers Steve)

As the our new Network Load Balancer (NLB) was not managed by us, they had not setup cookie based persistence. Or “Affinity” as described in the article below.

This was enabled and the problem went away.

It happens as the traffic from your browser may hit one WFE, and then another. A token is issued for each session so in this case setting cookie based persistence resolves the issue.



Resolution :
Enable the cookie based persistence on the NLB





Steve Peschka
Make Sure You Know This About SharePoint 2010 Claims Authentication - Sticky Sessions Are REQUIRED
http://blogs.technet.com/b/speschka/archive/2011/10/28/make-sure-you-know-this-about-sharepoint-2010-claims-authentication-sticky-sessions-are-required.aspx

Move SharePoint content to another Farm by backing up and restoring the SQL DB

At some point you will need to restore/move/backfill content from a Production Farm to a test or developer Farm.

You can do this by copying the SQL DB to your test SQL server and reattaching the DB in CA to the Target web application.

A few things to remember :

You will have to have the same solutions deployed on your target Farm.

Check the dependencies of any service applications i.e. Managed Metadata Service & others...

Make sure you have the same SQL server versions on target and source.

These instructions are for SQL Server 2008 R2 and SP 2010.

It is also possible to do this via PS, check the article at the end for further details.

On SQL Server

Get the current SQL permissions set on your target SQL DB.

Check the permissions set on the DB.

1.      SQL DB Properties > Files > Owner > record the account listed
2.      SQL DB Properties > Files > Permissions > record the accounts listed
3.      Security > Logins > accounts from step 2 >  Properties > User Mapping > select the target DB name > select source DB > record Permissions listed

Create a SQL DB backup of your Source web application DB.

Right click your target DB > Tasks > Backup > Backup type > select Full

Under Destination > Click remove > Click add > File name > browse … > Select your backup dir > enter a relevant filename > click ok > ok

Click OK to kick off your backup.

Copy the source SQL DB backup to your Target SQL Server.

To have a recovery option in case this doesn’t work. Backup your target SP web application DB either in SQL or via a SP Farm backup, both if you want to cover all bases.

On your SP Farm

At this point I normally stop the IIS sites on all SP WFE’s that use this target SQL DB to try stop connections to the DB’s. timer jobs may still kick off though.

On your target SP Farm, navigate to > Central Admin > application management > Databases > Manage content databases > Select you web application > select the target DB Name.

Select “Remove content DB” > OK

On SQL Server

On the target SQL server, restore the source DB over the Target DB

Check there are no connections to the source DB.

In SQL mgmt Studio, right click the server > Activity monitor > expand processes > filter by target db > right click and Kill Process on any connections.

Close the activity Monitor pane.

Right click your target DB > Tasks > Restore > Database.

Under To Database > ensure the target DB is selected.

Unser Source for restore select From Device > browse … > backup media File > Click Add > browse to backup location and source DB Backup file > click ok > Ok

Select the backup sets to restore > check the box

Select Options on the left > restore options > Overwrite the existing DB (with replace)

Click OK to kick off the restore.

Once completed, refresh the DB view, right click the restored DB > Files > Change the Owner to the correct account listed in step 1. Above (normally your SP farm account)

Click ok

Expand the target DB > select security > Users > remove any accounts from your source SP farm.

From the top level of SQL mgmt Studio, Select the server > Security > Logins > Select the equivalent account listed above in step 2. for your target SP Farm.

Right click > properties > user mapping > Users mapped to this login > select the target DB name > enable the permissions listed from step 3. above > click OK.

Your target DB should now have the same permissions set as before the restore.

On your SP Farm

On your target SP Farm, navigate to Central Admin > application management > Databases > Manage content databases > Select you web application > click “Add a Content DB”

Ensure the correct Web application and Database Server is listed.

Under “Database Name” Enter the target DB name.

Enter any configuration specifics, if any for the rest of the DB settings.

Click OK.

Your restored DB should now be attached successfully.

Start all IIS sites that use the Target DB’s is you stopped them.

Reset IIS on your WFE’s if you can.  

If you have a warm up script for your target web application, run this then access the site.

The site should eventually come up, if you got all your SQL permissions correct.

It can be really slow sometimes so be patient.

Natalia Tsymbalenko has some common errors in this useful article.

How restore a SharePoint 2010 content database on the different farm :

http://sharing-the-experience.blogspot.co.uk/2011/12/simple-concept-how-restore-sharepoint.html

Configure BLOB cache settings in SharePoint 2010

Blob cache is a quick and easy way to speed up your SharePoint site.

Here’s the MS blurb

A BLOB cache is a disk-based cache that stores binary large objects (BLOBs) such as frequently used image, audio, and video files, and other files that are used to display Web pages. Each front-end Web server maintains its own BLOB cache. When you enable a BLOB cache, you specify the file types to include in the cache and also the location of the BLOB cache. The first time that a BLOB file is requested, the file is copied from the database to the BLOB cache on the front-end Web server. Future requests to the front-end Web server for that same file are then served from the file that is stored in the BLOB cache, instead of being served from the database. This reduces the network traffic and the load on the database server.

To Enable Blob Cache

1. Make a copy of your web.config, open the live web.config

2.   Find the line <BlobCache location="" path="\.(gif|jpg|jpeg|jpe|jfif|bmp|dib|tif|tiff|ico|png|wdp|hdp|css|js|asf|avi|flv|m4v|mov|mp3|mp4|mpeg|mpg|rm|rmvb|wma|wmv)$" maxSize="10" enabled="false" />

3.  Change the location to a location that exists on each of your WFE’s. (Each WFE will have a cache dir, doesn’t have to be a share)

4.  Amend which files you want to cache by removing those you don’t want to cache.

5. Change the maxsize variable to something your drive location can cater for, size is in GB. (Note each IIS website will use this value, 3 sites = 30GB if set to 10)

6. Change the enabled attribute to “true” from “false”

7. Save the web.config file and load your site.

8. Check the blob cache gets created on each WFE.

To Flush the BLOB cache (SharePoint Server 2010)

When you flush the BLOB cache, you clear the contents of the BLOB cache for a Web application. This is useful if the BLOB cache becomes out of sync with the content. For example, after you restore a content database, the BLOB cache will be out of sync with the content. To correct that situation, you must flush the BLOB cache.

Create a PS script file containing the following.

$webApp = Get-SPWebApplication "<MyWebApplicationURL>"

[Microsoft.SharePoint.Publishing.PublishingCache]::FlushBlobCache($webApp)

Write-Host "Flushed the BLOB cache for:" $webApp

Save the file “FlushMyWebApplicationURL”, and test run it to see that the Blob Cache gets deleted.

Test show this  updates the files in the blobcache dir but doesn’t remove all the files entirely.

 More info on the MS site here : 

http://technet.microsoft.com/en-us/library/cc770229.aspx#BLOB

How to fix SharePoint Foundation event id 7043

On Windows 2008 R2, you may receive the following error message multiple times in the application event log.
This can be ignored or you can fix with the quick change below.

Error details :

Event ID: 7043

Load control template file /_controltemplates/TaxonomyPicker.ascx failed: Could not load type 'Microsoft.SharePoint.Portal.WebControls.TaxonomyPicker' from assembly 'Microsoft.SharePoint.Portal, Version=14.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c'.

To fix this :

1.Navigate to /14/TEMPLATE/ControlTemplates/TaxonomyPicker.ascx user control on each WFE server.
2.Open the user control in a text editor and locate the first line
3.Find the character string &#44; and replace with a comma ‘,’ (without quotes).
4.Save the user control

I have had occasions where the fix has been implemented but still get the error. Then fell back to the line in the MS article below.
"This message should be treated as log noise and can be ignored."

MS article :
http://support.microsoft.com/kb/2481844

Setting SQL Server Alias Names in SharePoint 2010

Setting up a SQL Alias name is useful if you have multiple farms in virtual environments. It helps as you always reference the same SQL alias and when you are moving DB's between different SQL Servers.

Ideally this needs to be done on all WFE & App servers, before installing SP and setting up your Farms.
Not required on your SQL server though.

1. Start “C:\windows\system32\cliconfg.exe”
Select the tab “Alias” and click the button “Add”.
Select “TCP/IP” and define the name of the alias "SPSQLSrv". The fill in the hostname of your database server and select the “Dynamically determine port”-checkbox. After that click “OK”.

If you running 64 bit, Start “C:\windows\syswow64\cliconfg.exe”
personally even though running 64 bit, i have had it working just using the 32bit version. never hurts to be thorough though.

2. Now on each SharePoint Server, create your new farm using the Sharepoint Products Configuration Wizard. Use the defined sql alias name as your database server. "SPSQLSrv"

Installing MS ForeFront Protection 2010 for SharePoint

Pre Req's : 

• Create a new Forefront admin account: SPForefrtadm (careful on account name length as have had problems)
  

• Make account SP farm admin (not good i know but req)

• Add account to server(s) local administrator & WSS_ groups.

• Add account to SQL server with roles dbcreator, public, securityadmin

One last thing  here, I got the following error when installing

This appeared in the event log : 

Insufficient SQL database permissions for user 'Name: domain\spforefrtadm SID: S-1-5-21-4128845870-1776218058-854377484-3634 ImpersonationLevel: None' in database 'SharePoint_Config_name' on SQL Server instance 'servername'. Additional error information from SQL Server is included below.

The SELECT permission was denied on the object 'Versions', database 'SharePoint_Config_name', schema 'dbo'.

• So looks like it requires dbo SQL permissions on the Config DB too.

This is so wrong MS! sort it out!!

To Install Forefront for SP ::

 Run forefrontsharepointsetup.exe  (needs to be installed on each server, one at a time)
Accept licence agreement > Next > Next
Change data folder locations to d:\path
Enter proxy details (if req)
Enter SP farm account (SPForeFntAdm) & PWD > Next
Do not use MS for updates, or do if you have a mgmt server > Next
Do not join customer experience > Next > Next
Click close to finish.
Run the "Forefront protection for SharePoint console"
Click “Activate now”
Enter the licence key : enter your lic
Enter agreement details ::
Licence Agreement number:
EXP DATE:

Repeat on each WFE\ Application servers.


To Configure the Forefront AV you may want to consider the following :
Select policy management > Antimalware > Realtime
Once installed, Set process count to "2", Maximum container scan time to "60" seconds and click "save". (this saves a huge number of threads being used and a long wait time if problems during file scans)

Note this next step is not required but you may want to consider to avoid each AV scanner chewing up your sever memory and frankly a bit of AV overkill.
Select policy management > Global Settings > Advanced options > Engine Management
Under "intelligent Engine Management" Select "Manual" and Disable all but "Microsoft Antimalware Engine" and click "save"



Run whole process above on each server in turn as the CA web services are restarted. So don’t run the installs all at the same time.

Once done with installs, on your central admin server navigate to

Central Admin > Security > Manage AV settings

Set your required settings for the Farm AV.

optional : 
Antivirus Time Out
something short unless you like waiting

Antivirus Threads
2 unless you have lots of memory

Extract all your Sharepoint Solutions to disk

A neat "how to" with a good explanation to extract all solutions from your SharePoint Farm.

Care of Shane Young :
https://msmvps.com/blogs/shane/archive/2011/05/05/using-powershell-to-export-all-solutions-from-your-sharepoint-2010-farm-and-other-fun.aspx

Thanks for clawing back hours from SharePoint and Dev hell

SharePoint Upgrade Failures

Recently I noticed that some servers in a Farm were listed as requiring an upgrade.
This process was used to resolve.
Don't forget to backup your Farm before trying anything.


Central Admin > Manage Servers in this Farm
Under status, two Servers were listed as "Upgrade required"

Reviewing the DB's showed that content DB's required an upgrade.
Central Admin > Upgrade and Migration > review DB status


To check further, from the hive folder run an upgrade check and pipe it to a file for review
stsadm -o localupgradestatus > c:\folder\upgradecheck.txt
Review the log file and find the section that Needs Upgrade
<status>Needs Upgrade</status>
This should list which components have an issue.


To resume a failed upgrade run the following, depending on your situation, on each server with the problem

psconfig.exe -cmd upgrade -inplace v2v -passphrase <insert-passphrase-here> -force
psconfig.exe -cmd upgrade -inplace b2b -passphrase <insert-passphrase-here> -force

v2v would be used for SharePoint 2007 databases placed in a SharePoint 2010 environment. 
For SharePoint 2010 databases in a newer patch level on a SharePoint 2010 environment you should use b2b
Check MS link below for psconfig ref on when to use -force option.



This still would not work and the error below occurred.

On step 3 I was getting the error "failed to create sharepoint timer service job to upgrade sharepoint products"
Logged in as the Setup account and re-ran the psconfig to resume the upgrade.
This time the following error occured.

####
An exception of type Microsoft.SharePoint.Administration.SPUpdatedConcurrencyException was thrown. Additional exception information: An update conflict has occurred, and you must re-try this action. The object SPUpgradeSession Name=Upgrade -20120320-184525-16 was updated by DOMAIN\ACCOUNT, in the PSCONFIG (7240) process, on machine SERVER. View the tracing log for more information about the conflict.

Total number of configuration settings run: 3

Total number of successful configuration settings: 2

Total number of unsuccessful configuration settings: 1

Successfully stopped the configuration of SharePoint Products.

Configuration of SharePoint Products failed. Configuration must be performed before you use SharePoint Products. For further details, see the diagnostic log located at [LOCATION OF LOG] and the application event log.
####

Sharepointgeoff advises to Resolve:

1. stsadm -o setproperty -pn command-line-upgrade-running -pv No
2. IISReset
3. Restart the Windows SharePoint Timer (cmd window : net stop sptimerv4, then net start sptimerv4)
4. psconfig –cmd upgrade –inplace b2b –wait –force (or v2v, whatever was run above)


Then ran the SP wizard on the servers in question and voila, happy SP servers.




Psconfig command-line reference (SharePoint Server 2010)
http://technet.microsoft.com/en-us/library/cc263093.aspx

Resume upgrade (SharePoint Server 2010)
http://technet.microsoft.com/en-us/library/ff382638.aspx

Relevant info ::
In some cases, you might have to restart upgrade to finish upgrading your sites from Microsoft Office SharePoint Server 2007 to Microsoft SharePoint Server 2010. For example:

During an in-place upgrade, if the server restarts or the upgrade fails, you must restart the upgrade process by using Psconfig.exe to upgrade the remaining sites.


During a database attach upgrade, any sites that cannot be upgraded will be skipped. After you have corrected any issues in the sites (such as a missing template or language pack, or the site being set to read-only or having exceeded its quota), you can restart upgrade by using a Windows PowerShell command to upgrade just the skipped sites.

SharepointGeoff : SPUpdatedConcurrencyException Fix when trying to install June 2011 CU for SharePoint Server
http://www.sharepointgeoff.com/spupdatedconcurrencyexception-fix-when-trying-to-install-june-2011-cu-for-sharepoint-server/