kenaflow – Enterprise Business Automation with PowerShell

Almost exactly four years ago I started developing the software kenaflow. The project started as a workflow engine for SharePoint. Meanwhile the software can do a lot more, e.g. workflows for email inboxes.

Workflows with kenaflow work via PowerShell scripts: The data to be processed is passed individually to one or more scripts. What the developer does there is his/her business.

kenaflow creates the framework and takes care of the execution (scheduling).

kenaflow provides HTTPS endpoints, e.g. for web hooks or SharePoint remote events.

The whole thing is mega flexible and scalable.

Interested? Then feel free to send me a message! – kenaflow is free for the first six months in the full version!

The latest offering is kenaflow as a hosted cloud service. We provide kenaflow on a dedicated virtual machine per customer. You develop workflow scripts via Visual Studio Code Remote – using SSH. – Even on the smallest virtual machine you can run an unlimited number of workflows, limited only by the computational intensity and execution frequency of your workflows.

“Access Denied” When Trying to Save a Web as Template in SP 2019 On-Premises

My customer tries to save a sub web “as template” using “/_layouts/15/savetmpl.aspx“.

He gets an “Access Denied” (“This site has not been shared with you”) page.

It took me a while to understand the ULS (SharePoint log). Finally I found the related lines.

Requiring ACPRight

The permission is being checked for a write operation

Requiring ManageListsRight and ACPRight as this is a write operation on catalog

Permission check failed. Asking for 0x00040802, have 0x7FFFFFFFFFFBFFFF

SPRequest.PutFile: UserPrincipalName=i:0).w|s-1-5-21-45689356773-24675323-1484686, AppPrincipalName= ,bstrUrl=https://sharepoint.farm ,bstrWebRelativeUrl=_catalogs/solutions/ikarstein.wsp ,cbFile=11111 ,punkSPFileMgr=<null> ,punkFFM=<null>

System.UnauthorizedAccessException: Access denied., StackTrace:   
 at Microsoft.SharePoint.SPFileCollection.AddStreamOrBytesInternal(String urlOfFile, Stream file, Int64 fileSizeToSave, SPFileStreamManager spmgr, Int64 fileOpt, String createdBy, String modifiedBy, Int32 createdByID, Int32 modifiedByID, DateTime timeCreated, DateTime timeLastModified, Object varProperties, String checkInComment, Stream formatMetadata, String lockIdMatch, String etagToMatch, SPLockType lockType, String lockId, TimeSpan lockTimeout, Boolean validateRequiredFields, Guid bitsSessionId, Guid originatorId, SPVirusCheckStatus& virusCheckStatus, String& virusCheckMessage, String& etagNew, Boolean& ignoredRequiredProps, SPFileInfo& fileProps)    
 at Microsoft.SharePoint.SPFileCollection.Add(String urlOfFile, Stream file, Hashtable properties, Boolean overwrite, Boolean requireWebFilePermissions)    
 at Microsoft.SharePoint.SPSolutionExporter.ExportWebToGallery(SPWeb web, String solutionFileName, String title, String description, ExportMode exportMode, Boolean includeContent, String workflowTemplateName, String destinationListUrl, Action`1 solutionPostProcessor, Boolean activateSolution)    
 at Microsoft.SharePoint.ApplicationPages.SaveAsTemplatePage.BtnSaveAsTemplate_Click(Object sender, EventArgs e)    
 at System.Web.UI.WebControls.Button.OnClick(EventArgs e)    
 at System.Web.UI.WebControls.Button.RaisePostBackEvent(String eventArgument)    
 at System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)    
 at System.Web.UI.Page.ProcessRequest(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)    
 at System.Web.UI.Page.ProcessRequest()    
 at System.Web.UI.Page.ProcessRequest(HttpContext context)    
 at System.Web.HttpApplication.CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()    
 at System.Web.HttpApplication.ExecuteStepImpl(IExecutionStep step)    
 at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously)    
 at System.Web.HttpApplication.PipelineStepManager.ResumeSteps(Exception error)    
 at System.Web.HttpApplication.BeginProcessRequestNotification(HttpContext context, AsyncCallback cb)    
 at System.Web.HttpRuntime.ProcessRequestNotificationPrivate(IIS7WorkerRequest wr, HttpContext context)    
 at System.Web.Hosting.PipelineRuntime.ProcessRequestNotificationHelper(IntPtr rootedObjectsPointer, IntPtr nativeRequestContext, IntPtr moduleData, Int32 flags)    
 at System.Web.Hosting.PipelineRuntime.ProcessRequestNotification(IntPtr rootedObjectsPointer, IntPtr nativeRequestContext, IntPtr moduleData, Int32 flags)    
 at System.Web.Hosting.UnsafeIISMethods.MgdIndicateCompletion(IntPtr pHandler, RequestNotificationStatus& notificationStatus)    
 at System.Web.Hosting.UnsafeIISMethods.MgdIndicateCompletion(IntPtr pHandler, RequestNotificationStatus& notificationStatus)    
 at System.Web.Hosting.PipelineRuntime.ProcessRequestNotificationHelper(IntPtr rootedObjectsPointer, IntPtr nativeRequestContext, IntPtr moduleData, Int32 flags)    
 at System.Web.Hosting.PipelineRuntime.ProcessRequestNotification(IntPtr rootedObjectsPointer, IntPtr nativeRequestContext, IntPtr moduleData, Int32 flags)

Access Denied. Exception: Access denied., StackTrace:  
 at Microsoft.SharePoint.Library.SPRequestInternalClass.PutFile(String bstrUrl, String bstrWebRelativeUrl, Object punkFile, Int64 cbFile, Object punkSPFileMgr, Object punkFFM, SPFileSaveParams sfsp, SPFileInfo& pFileProps, UInt32& pdwVirusCheckStatus, String& pVirusCheckMessage, String& pEtagReturn, Byte& piLevel, Int32& pbIgnoredReqProps)    
 at Microsoft.SharePoint.Library.SPRequest.PutFile(String bstrUrl, String bstrWebRelativeUrl, Object punkFile, Int64 cbFile, Object punkSPFileMgr, Object punkFFM, SPFileSaveParams sfsp, SPFileInfo& pFileProps, UInt32& pdwVirusCheckStatus, String& pVirusCheckMessage, String& pEtagReturn, Byte& piLevel, Int32& pbIgnoredReqProps).

It took me a lot of time to figure that out…

$w = Get-SPWeb "https://sharepoint.farm"

$w.site.DenyPermissionsMask

([long]$w.site.DenyPermissionsMask).ToString("x")

0x7FFFFFFFFFFBFFFFL

0x7FFFFFFFFFFFFFFFL

0x7FFFFFFFFFFFFFFFL -bxor 0x40000

The line ” $w.site.DenyPermissionsMask ” returned the result: “AddAndCustomizePages” => A (A) C P => ACP ? => ACPRight … ?? There it was !!!

Hex: 0x40000 (line ([long]$w.site.DenyPermissionsMask).ToString("x") )

The long hex values showed me that exactly that right was for some reasons “denied” on the site collection.

I solved it using this command:

$w.site.DenyPermissionsMask = [microsoft.sharepoint.spbasepermissions]::EmptyMask

The previous setting can be restored with:

$w.site.DenyPermissionsMask =[microsoft.sharepoint.spbasepermissions]::AddAndCustomizePages

After the fix, I checked all site collections for this particular setting:

get-spsite -limit all | % {
    if( ($_.DenyPermissionsMask) -ne "EmptyMask") {
        write-host $_.url  -ForegroundColor red
        write-host "`t" ($_.DenyPermissionsMask) -ForegroundColor red
    } else {
        #write-host $_.url  -ForegroundColor green
    }
}

… No other site collection has had a setting other than “EmptyMask”.

SQL Execution in Powershell: Write Server Messages to Console

It has been a problem for me to work with SQL backup and restore for SharePoint with Powershell…

Issuing the commands is no problem. A simple SQL Server connection is required.

But for long running tasks I need to see the server messages in Powershell.

Here is a trick to do that. I use it since years. As a C# developer I know the “event handler” that can receive the messages…

$cnn = New-Object System.Data.SqlClient.SqlConnection

$cnn.add_InfoMessage([System.Data.SqlClient.SqlInfoMessageEventHandler] {
  param($sender, $event);

  Write-Host $event.Message;
}); 

$cnn.FireInfoMessageEventOnUserErrors = $true

$cnn.ConnectionString = "Server=sqlserver;Database=master;Integrated Security=True;Connection Timeout=86400"

$cnn.Open()

“add_InfoMessage” does the trick.

Cleanup IIS Log using PowerShell

On every SharePoint or IIS server I need the following script:

Short Version, e.g. for Windows Task Scheduler

powershell -command “get-childitem -path c:\inetpub\logs\logfiles -filter *.log -recurse | ? { $_.LastWriteTimeUtc -lt [datetime]::utcnow.addmonths(-2)} | remove-item -force -confirm:$false -whatif”

(You may need to adjust the path to the log files!)

Long Version

cls

$path = "C:\inetpub\logs"
 
$dt = [datetime]::Now

$s = [long]0
Get-ChildItem "$($path)\LogFiles" -Filter "*.log" -Recurse | % { $s += $_.Length }

Write-host "Size before: $($s / 1024 / 1024) MB"

$s = [long]0
Get-ChildItem "$($path)\LogFiles" -Filter "*.log" -Recurse | ? { ($dt - $_.LastWriteTime).TotalDays -gt 30 } | % { $s += $_.Length; $_ | Remove-Item -Confirm:$false -Force }

Write-host "Removed: $($s / 1024 / 1024) MB"

$s = [long]0
Get-ChildItem "$($path)\LogFiles" -Filter "*.log" -Recurse | % { $s += $_.Length }

Write-host "Size after: $($s / 1024 / 1024) MB"

New-SPConfigurationDatabase – Error “Given key was not present in the directory”

During configuration of a new SharePoint 2019 farm I got an error in PowerShell Cmdlet New-SPConfigurationDatabase.

The given key was not present in the directory.

I found some hints… E.g.: http://alstechtips.blogspot.com/2014/04/sharepoint-2013-given-key-was-not.html

I checked the farm account used with the Cmdlet.

I found that the farm account did not have “Read” permission for “All Authenticated Users”.

After granting this permission the Cmdlet succeeded.

Set TLS Version in PowerShell

It has been described lots of times before. It’s just for the records…

This code sets the used TLS version for secure HTTP connections to version 1.2

[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12

Here are the valid values:

SystemDefault
Ssl3
Tls
Tls11
Tls12

The latests supported version depends on your .NET Framework. On Windows 10 20H2 the setting Tls13 will work too.

With this code you can list the valid values:

[Net.SecurityProtocolType]::GetNames([Net.SecurityProtocolType])

Create Self Signed Certificate with PowerShell on Windows Server 2012 R2.

A little bit outdated. I know.

But customers not always have the latest server versions running.

Today a customer asked me how to create a self signed SSL certificate with “subject alternate names” (SAN) using PowerShell on Windows Server 2012 R2.

The default cmdlet “New-SelfSignedCertificate” has not all features on this server OS.

The customer found a script but it did not offer SAN. I found another one (in german language) that gave me the other information…

Sources:

https://docs.microsoft.com/en-us/archive/blogs/vishalagarwal/generating-a-certificate-self-signed-using-powershell-and-certenroll-interfaces

https://www.symplasson.de/it-blog/san-erweiterung-csr-windows-ca

I assembled both scripts. Here is the result.

# script assembled from these sources
#  - https://docs.microsoft.com/en-us/archive/blogs/vishalagarwal/generating-a-certificate-self-signed-using-powershell-and-certenroll-interfaces
#  - https://www.symplasson.de/it-blog/san-erweiterung-csr-windows-ca
# …by Ingo Karstein ( ik a.t. kenaro.com)
#
# Useful e.g. on Windows Server 2012R2 because there are less functionality in cmdlet New-SelfSignedCertificate.
#
# The SSL cert ist written to certstore "My" of "LocalMachine"

 $name = new-object -com "X509Enrollment.CX500DistinguishedName.1"
 $name.Encode("CN=srv", 0)

 $key = new-object -com "X509Enrollment.CX509PrivateKey.1"
 $key.ProviderName = "Microsoft RSA SChannel Cryptographic Provider"
 $key.KeySpec = 1
 $key.Length = 4096
 $key.SecurityDescriptor = "D:PAI(A;;0xd01f01ff;;;SY)(A;;0xd01f01ff;;;BA)(A;;0x80120089;;;NS)"
 $key.MachineContext = 1
 $key.Create()

 $serverauthoid = new-object -com "X509Enrollment.CObjectId.1"
 $serverauthoid.InitializeFromValue("1.3.6.1.5.5.7.3.1")

 $AlternativeNames=@("srv", "localhost")
 $AlternativeIPs=@("127.0.0.1", "::1")

 $SAN = New-Object -ComObject X509Enrollment.CX509ExtensionAlternativeNames
 $IANs = New-Object -ComObject X509Enrollment.CAlternativeNames

 foreach ($SANstr in $AlternativeNames)
 {
     $IAN = New-Object -ComObject X509Enrollment.CAlternativeName
     $IAN.InitializeFromString(0x3,$SANstr)
     $IANs.Add($IAN)
 }

 foreach ($SANstr in $AlternativeIPs)
 {
     $IAN = New-Object -ComObject X509Enrollment.CAlternativeName
     $IAN.InitializeFromString(0x3,$SANstr)
     $IANs.Add($IAN)
     $IAI = New-Object -ComObject X509Enrollment.CAlternativeName    
     $IAI.InitializeFromRawData(8, 0x1, 
        [Convert]::ToBase64String(
       ([System.Net.IpAddress] $SANstr).GetAddressBytes())) $IANs.Add($IAI)
 }

 $SAN.InitializeEncode($IANs)

 $ekuoids = new-object -com "X509Enrollment.CObjectIds.1"
 $ekuoids.add($serverauthoid)

 $ekuext = new-object -com "X509Enrollment.CX509ExtensionEnhancedKeyUsage.1"
 $ekuext.InitializeEncode($ekuoids)

 $cert = new-object -com "X509Enrollment.CX509CertificateRequestCertificate.1"
 $cert.InitializeFromPrivateKey(2, $key, "")
 $cert.Subject = $name
 $cert.Issuer = $cert.Subject
 $cert.NotBefore = (get-date).AddDays(-1)
 $cert.NotAfter = $cert.NotBefore.Addyears(50)
 $cert.X509Extensions.Add($ekuext)
 $cert.X509Extensions.Add($SAN)
 $cert.Encode()

 $enrollment = new-object -com "X509Enrollment.CX509Enrollment.1"
 $enrollment.InitializeFromRequest($cert)

 $certdata = $enrollment.CreateRequest(0)

 $enrollment.InstallResponse(2, $certdata, 0, "")

SharePoint On-Prem Search Service Error: “Access Denied”

I got this error:

Search Service Application: Crawl Log – Error Breakdown

Access is denied. Verify that either the Default Content Access Account has access to this repository, or add a crawl rule to crawl this repository. If the repository being crawled is a SharePoint repository, verify that the account you are using has “Full Read” permissions on the SharePoint Web Application being crawled.

…but the “default content access account” already had “Full Read” permission on the web application.

  1. Check “DisableLoopbackCheck” in Windows Registry on SharePoint Server.
  2. Or check “BackConnectionHostNames” in Windows Registry on SharePoint Server.
  3. Check the content source setting: Is the web application url the one defined as “Default Zone” in Alternate Access Mappings of the web application?
  4. Check the authentication provider for the web application: SharePoint Search works with Windows Integrated Authentication. It is required that the “default zone” is configured with “Windows Integrated Authentication”, not “Basic Auth”!

Maybe the server requires a reboot after changing the configuration.

SharePoint Error: Metadata backend service request ExpectedFailure: Metadata backend service request failed in a authorized access exception: System.ServiceModel.FaultException`1[System.ServiceModel.ExceptionDetail]: AccessDeniedEx:The current user has insufficient permissions to perform this operation.

I got the following error:

Exception returned from back end service. System.ServiceModel.FaultException`1[System.ServiceModel.ExceptionDetail]: AccessDeniedEx:The current user has insufficient permissions to perform this operation. (Fault Detail is equal to An ExceptionDetail, likely created by IncludeExceptionDetailInFaults=true, whose value is: System.UnauthorizedAccessException: AccessDeniedEx:The current user has insufficient permissions to perform this operation.
at Microsoft.SharePoint.Taxonomy.MetadataWebServiceApplication.CheckPermission(SPIisWebServiceApplicationRights requiredPermissions)
at Microsoft.SharePoint.Taxonomy.TaxonomyDatabaseMapper.GetTermSetWithAllTerms(Guid rawPartitionId, Guid termSetGuid, Boolean includeDeprecated, Boolean limitMembership)
at Microsoft.SharePoint.Taxonomy.MetadataWebServiceApplication.GetTermSetWithAllTerms(Guid rawPartitionId, Guid termSetGuid, Boolean includeDeprecated, Boolean limitMembership)
at SyncInvokeGetTermSetWithAllTerms(Object , Object[] , Object[] )
at System.ServiceModel.Dispatcher.SyncMethodInvoker.Invoke(Object instance, Object[] inputs, Object[]& outputs)
at System.ServiceModel.Dispatcher.DispatchOperationRuntime.InvokeBegin(MessageRpc& rpc)
at System.ServiceModel.Dispatche…).

Metadata backend service request ExpectedFailure: Metadata backend service request failed in a authorized access exception: System.ServiceModel.FaultException`1[System.ServiceModel.ExceptionDetail]: AccessDeniedEx:The current user has insufficient permissions to perform this operation. (Fault Detail is equal to An ExceptionDetail, likely created by IncludeExceptionDetailInFaults=true, whose value is: System.UnauthorizedAccessException: AccessDeniedEx:The current user has insufficient permissions to perform this operation.
at Microsoft.SharePoint.Taxonomy.MetadataWebServiceApplication.CheckPermission(SPIisWebServiceApplicationRights requiredPermissions)
at Microsoft.SharePoint.Taxonomy.TaxonomyDatabaseMapper.GetTermSetWithAllTerms(Guid rawPartitionId, Guid termSetGuid, Boolean includeDeprecated, Boolean limitMembership)
at Microsoft.SharePoint.Taxonomy.MetadataWebServiceApplication.GetTermSetWithAllTerms(Guid rawPartitionId, Guid termSetGuid, Boolean includeDeprecated, Boolean limitMembership)
at SyncInvokeGetTermSetWithAllTerms(Object , Object[] , Object[] )
at System.ServiceModel.Dispatcher.SyncMethodInvoker.Invoke(Object instance, Object[] inputs, Object[]& outputs)
at System.ServiceModel.Dispatcher.DispatchOperationRuntime.InvokeBegin(MessageRpc& rpc)
at System.ServiceModel.Dispatche…).

Solution

If you are running your Search Service Application using a different account than the farm account then you need to grant “Full Control” permission the this account on the Managed Metadata Service Application!

SharePoint: Error in ULS and Windows Event Log: Unknown SQL Exception 241 occurred. Additional error information from SQL Server is included below.

Today I did some SharePoint 2016 => 2019 migration tests for a customer.

I set up the new farm weeks ago, but today I saw the following error for the first time in the SharePoint ULS and Windows Event Log:

Entering Monitored Scope (Storage Metrics processing timer job). Parent=Timer Job job-storage-metrics-processing

System.Data.SqlClient.SqlException (0x80131904): Fehler beim Konvertieren einer Zeichenfolge in ein Datum und/oder eine Uhrzeit.
at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection, Action1 wrapCloseInAction) at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose) at System.Data.SqlClient.TdsParser.TryRun(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj, Boolean& dataReady) at System.Data.SqlClient.SqlCommand.FinishExecuteReader(SqlDataReader ds, RunBehavior runBehavior, String resetOptionsString, Boolean isInternal, Boolean forDescribeParameterEncryption, Boolean shouldCacheForAlwaysEncrypted) at System.Data.SqlClient.SqlCommand.RunExecuteReaderTds(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, Boolean async, Int32 timeout, Task& task, Boolean asyncWrite, Boolean inRetry, SqlDataReader ds, Boolean describeParameterEncryptionRequest) at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method, TaskCompletionSource1 completion, Int32 timeout, Task& task, Boolean& usedCache, Boolean asyncWrite, Boolean inRetry)
at System.Data.SqlClient.SqlCommand.InternalExecuteNonQuery(TaskCompletionSource`1 completion, String methodName, Boolean sendToPipe, Int32 timeout, Boolean& usedCache, Boolean asyncWrite, Boolean inRetry)
at System.Data.SqlClient.SqlCommand.ExecuteNonQuery()
at Microsoft.SharePoint.Utilities.SqlSession.ExecuteNonQuery(SqlCommand command, SqlInfoMessageEventHandler handler) ClientConnectionId:c66450bf-adc7-4fc4-bbcd-e9e2c050a75a Error Number:241,State:1,Class:16

Unknown SQL Exception 241 occurred. Additional error information from SQL Server is included below. Fehler beim Konvertieren einer Zeichenfolge in ein Datum und/oder eine Uhrzeit.

It happens during execution of the SP timer job “Storage Metrics Processing” (technical name: “job-storage-metrics-processing”).

In the “job history” in the Central Administration I could see the affected database. It was one of the migrated database I mounted before to the new SP farm.

Fix

After two hours I found the solution:

In SQL Server the login for the new farm account “sp-farm-new” was set to DEFAULT LANGUAGE = GERMAN.

I changed the DEFAULT LANGUAGE of all accounts to “ENGLISH” and the error was gone.

I checked all SP service accounts of the old farm: They are all set to DEFAULT LANGUAGE = ENGLISH.