This blog explains how to take the standard examples for Dynamics 365 for Finance and Operations integration from Github and authenticate to an on-premise instance of Finance and Operations. At the end you'll also find some troubleshooting tips if it doesn't work first time, which can be useful for any scenario where something is trying to authenticate to services.

Environment prerequisites

There are a few items required before you start:
- Installed Visual Studio 2017 Enterprise edition (edition appears to not be important)
- Downloaded the examples from GitHub: https://github.com/Microsoft/Dynamics-AX-Integration
- Open the ServicesSamples.sln solution
- Within Visual Studio, go to Tools->NuGet package manager-> manage NuGet packages for solution, there it recognises there are some missing – click restore in the top right and it downloads them automatically.

ADFS Setup

Now on ADFS server in my on-prem environment, I need to add a client application. From “AD FS Management” I see Application Groups, open the default application group for Dynamics 365 called “Microsoft Dynamics 365 for Operations On-Premises”, it will look something similar to this:

Click “Add application…” at the bottom, add a new server application:

Add the redirect URL, this should be the URL for the D365 application, take a note of the client identifier as you’ll need to use this in your client application later:

Select to generate a shared secret – you must copy this now, as it will not be shown again – your client app needs this detail to connect to D365:

Next, back in the application group window, edit the “Microsoft Dynamics 365 for Operations On-premises – Web API” item:

On the “Client permissions” tab add a new record for the server application created in the previous step:

Code

Within the ServicesSamples.sln solution open the ClientConfiguration.cs source file and modify similar to below (using the values from your ADFS configuration above):

public static ClientConfiguration OneBox = new ClientConfiguration()
{
UriString = "https://ax.d365ffo.zone1.saonprem.com/namespaces/AXSF/", //the normal URL for logging into D365
UserName = "not used",
Password = "",

//Note that AOS config XML is on AOS machines in: C:\ProgramData\SF\AOS_10\Fabric\work\Applications\AXSFType_App84\AXSF.Package.1.0.xml

ActiveDirectoryResource = "https://ax.d365ffo.zone1.saonprem.com/", //this is the value for AADValidAudience from the AOS config xml
ActiveDirectoryTenant = "https://dax7sqlaoadfs1.saonprem.com/adfs",//this is the value for AADIssuerNameFormat (minus the placeholder {0}, instead suffix "/adfs") from AOS config xml
ActiveDirectoryClientAppId = "6c371040-cf6b-4154-b9c4-75e613fb5104", //client app ID is from ADFS management - configure a application group
ActiveDirectoryClientAppSecret = "MO-tVemKqAjVLj1NdcCs3mfiWw2X3ZNyjuFe0UYg", //secret is from ADFS management - same place as the client app ID

// Change TLS version of HTTP request from the client here
// Ex: TLSVersion = "1.2"
// Leave it empty if want to use the default version
TLSVersion = "",
};

AX Setup

Also need to add the application in AX application too, under System administration > setup > Azure Active Directory applications, using the client ID you put into your client code:

Troubleshooting:

ADFS group creation fails

If ADFS group creation fails as shown below with the error MSIS7613 Each identifier must be unique across all relying party trusts in AD FS configuration. This means that the URL entered for Web API is already registered in another group – probably the default D365 group. To resolve this see below.

Locate the standard Microsoft Dynamics 365 for Operations On-premises ADFS application group and open it.

Forbidden

The error below occurs if the setup within AX hasn’t been completed within the AX application under System administration > setup > Azure Active Directory applications. The error below was reported back to the calling client application.
0:025> !pe
Exception object: 0000029e2b48e1d8
Exception type: System.ServiceModel.Web.WebFaultException`1[[System.ComponentModel.Win32Exception, System]]
Message: Forbidden
InnerException: <none>
StackTrace (generated):
<none>
StackTraceString: <none>
HResult: 80131501
0:025> !clrstack
OS Thread Id: 0x1cb4 (25)
Child SP IP Call Site
000000f0381bc3e8 00007ff86e233c58 [HelperMethodFrame: 000000f0381bc3e8]
000000f0381bc4d0 00007ff808fe8642 Microsoft.Dynamics.Ax.Services.ServicesSessionProvider.ThrowSessionCreationException(Microsoft.Dynamics.Ax.Services.ServicesSessionCreationErrorCode)
000000f0381bc520 00007ff808fe45b0 Microsoft.Dynamics.Ax.Services.ServicesSessionProvider.GetSession(Boolean, Boolean, System.String, System.String, System.String, System.String, System.Security.Claims.ClaimsIdentity)
000000f0381bc690 00007ff808fe4014 Microsoft.Dynamics.Ax.Services.ServicesSessionManager.InitThreadSession(Boolean, Microsoft.Dynamics.Ax.Xpp.AxShared.SessionType, Boolean, System.String, System.String, System.String, System.String, System.Security.Claims.ClaimsIdentity)
000000f0381bc730 00007ff808fe3ea6 Microsoft.Dynamics.Platform.Integration.Common.SessionManagement.ServicesAosSessionManager.InitializeSession(Boolean, System.String, System.Security.Claims.ClaimsIdentity)
000000f0381bc7a0 00007ff808fe366a Microsoft.Dynamics.Platform.Integration.Common.SessionManagement.OwinRequestSessionProvider.CreateSession(System.Security.Claims.ClaimsIdentity)
000000f0381bc7f0 00007ff808fe34cc Microsoft.Dynamics.Platform.Integration.Common.SessionManagement.ServicesRequestSessionHelper.EnsureRequestSession(Microsoft.Dynamics.Platform.Integration.Common.SessionManagement.IServicesRequestSessionProvider, System.Security.Claims.ClaimsIdentity)
000000f0381bc830 00007ff808fe2a86

Audience validation failed

The error below occurs if the value you’re using in your client application for ActiveDirectoryResource (from ClientConfiguration.cs in the example apps) doesn’t match the value in the AOS configuration for AADValidAudience. The AOS configuration is here: C:\ProgramData\SF\AOS_10\Fabric\work\Applications\AXSFType_App84\AXSF.Package.1.0.xml
Note that the error passed back to the client application is not as detailed as this – this error was from catching the exception directly on the AOS machine using WinDbg.

0:029> !pe
Exception object: 00000166f07da608
Exception type: System.IdentityModel.Tokens.SecurityTokenInvalidAudienceException
Message: IDX10214: Audience validation failed. Audiences: 'http://tariqapp.saonprem.com'. Did not match: validationParameters.ValidAudience: 'null' or validationParameters.ValidAudiences: 'https://ax.d365ffo.zone1.saonprem.com, 00000015-0000-0000-c000-000000000000, https://ax.d365ffo.zone1.saonprem.com/'
InnerException: <none>
StackTrace (generated):
<none>
StackTraceString: <none>
HResult: 80131501

Strong name validation failed on first client application run

If you tried to run OdataConsoleApplication and it failed with error: Could not load file or assembly 'Microsoft.OData.Client, Version=6.11.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35' or one of its dependencies. Strong name validation failed. (Exception from HRESULT: 0x8013141A)

The root cause of this could be that it’s looking for 6.11 but 6.15 is the version installed – from the NuGet package manager in VS you can change the version, then build, success, then run, success.

ADFS error log

To help troubleshoot ADFS errors you can view the event viewer on the ADFS server, as shown below, it’s under Applications and services logs > AD FS > Admin.

Sometimes when services are trying to authenticate to an AOS in Dynamics 365 for Finance and Operations, both in the Cloud version and the on-premise version, the calling application may receive the error message "forbidden" back from the AOS. This message is deliberately vague, because we don't want a calling application to be able to poke the AOS and learn about how to get in, but unfortunately that vagueness can make it difficult to figure out what is actually wrong, in this post we'll discuss what's happening in the background and how to approach troubleshooting.

Anything which is calling web services could receive this "Forbidden" error - for example an integrated 3rd party application, or Financial Reporting (formerly Management Reporter).

First let's talk about how authentication to Finance and Operations works, there are two major stages to it:

1. Authentication to AAD (in Cloud) or ADFS (in on-premise)- this is happening directly between the caller and AAD/ADFS - the AOS isn't a part of it.
2. Session creation on the AOS - here the caller is giving the token from AAD/ADFS to the AOS, then AOS attempts to create a session.

The "forbidden" error occurs during the 2nd part of the process - when the AOS is attempting to create a new session. The code within the AOS which does this has a few specific cases when it will raise this:

- Empty user SID
- Empty session key
- No such user
- User account disabled
- Cannot load user groups for user

For all of these reasons the AOS is looking at the internal setup of the user in USERINFO table - it's not looking at AAD/ADFS. In a SQL Server based environment (so Tier1 or on-premise) you can run SQL Profiler to capture the query it's running against the USERINFO table and see what it's looking for.

Examples:

- Financial Reporting (Management reporter) might report "Forbidden" if the FRServiceUser is missing or incorrect in USERINFO. This user is created automatically, but could have been modified by an Administrator when trying to import users into the database.
- When integrating 3rd party applications if the record in "System administration > setup > Azure Active Directory applications" is missing

There are some features within Dynamics 365 for Finance and Operations on-premise which rely on an internet connection.

This means that the on-premise version DOES by default have a dependency on some cloud services - BUT you can turn that off, so there is no dependency.

As an example, last week there was an AAD outage, which affected on-premise customer's ability to log into the application. What was happening was - you'd log in as normal - see the home page for a moment, then it would redirect to the AAD login page, which was down, so the user would be stuck.

In the background this relates to the Skype presence feature - after the user logs in, in the background the system is contacting the Skype service online - which is what triggers that redirect to AAD when AAD is unavailable.

There is a hotfix available which allows a System Administrator to turn off all cloud/internet related functions in the on-prem version, details are available here:
Disable internet connectivity

In Dynamics 365 for Finance and Operations the document management feature allows you to attach documents (files and notes) to records within the application. There are several different options for storage of those documents – in this document we will explain the advantages and disadvantages of each option

Document storage locations

There are 3 possible options for document storage:

• Azure storage: In the cloud version of Finance and operations this will store documents in Azure blob storage, in the on-premise version this will store documents in the file share given in the environment deployment options in LCS*
• Database: stores documents in the database
• SharePoint: stores documents in SharePoint Online, this is currently only supported for the cloud version. Support for on-premises SharePoint is planned to be added in the future

Each document storage option can be configured per document type– meaning that it’s possible to configure a type of document “scanned invoices” and choose storage “Database”, and configure another type of document “technical drawings” and choose storage “Azure storage”.

Classes

When configuring document types there are 3 different classes of document available, each class of document only allows certain storage locations:
- Attach file: this allows selection of “Azure storage” or “SharePoint” locations
- Attach URL: this allows only “Database” location
- Simple note: this allows only “Database” location

Azure storage

This type of storage can be configured for the “attach file” class of document only.

As mentioned earlier in this document, in the cloud version of Finance and operations this will store documents in Azure blob storage, in the on-premise version this will store documents in the file share given in the environment deployment options in LCS.

In the cloud version the an Azure storage account is automatically created when an environment is deployed. No direct access to the storage account is given, access is only via the application. This is a highly available geo-replicated account, so there are no additional considerations required to ensure business continuity for this component.

In the on-premise version an SMB 3.0 file share is specified at environment deployment time. High availability and disaster recovery options must be considered to ensure availability of this file share, the application accesses this using its UNC path – ensure this UNC path is available at all times.

Files stored in this way are not readable by directly accessing the file share location – they are intended only to be accessed through Finance and Operations – specifically files stored will be renamed to a GUID type name, and their file extension is removed. Within Finance and Operations a database table provides the link between the application and the file stored on the file system.
No direct access to this folder should be allowed for users, access for the Finance and Operations server process is controlled through the certificate specified during environment deployment.

Database

Database storage will be used automatically for document types using classes “Attach URL” or “Simple note”. The “Attach file” class of documents will not be stored in the database.
Documents stored in the database will be highly available by virtue of the SQL high availability options which are expected to be in place already as a requirement of Finance and Operations.

SharePoint

This type of storage can be configured for the “Attach file” class of document only.

For the cloud version of Finance and Operations, SharePoint Online is supported but currently SharePoint on-premise is not supported. For the on-premise version SharePoint Online is also not supported currently.

SharePoint Online is a highly available and resilient service, we recommend to review our documentation for more information.

Cloud versus On-premise

In the cloud version of Finance and Operations, for file storage, either SharePoint Online or Azure blob storage can be used.
In the on-premise version, for file storage, only Azure blob storage can be used – which will store files in a network file share as defined in the environment deployment options.
*The screenshot below shows the setting for file share storage location used by on-premise environments when selecting “Azure storage”.

This document contains tips for troubleshooting on-premise Dynamics 365 for Finance and Operations environment deployment failures, based on my own experiences when troubleshooting this process for the first time.

Types of failures

The first type of failure I am seeing here is a simple redeploy of the environment. Originally I was trying to deploy a custom package, but it failed and I didn’t know why, so I deleted the environment and was redeploying with vanilla – no custom bits, just the base, and it still failed. In LCS after it runs for approx. 50 minutes I see the state change to Failed. There is no further log information in LCS itself, that information is within the respective machines in the on-premise environment.

Orchestrators

The Orchestrator machines trigger the deployment steps. In the base topology there are 3 orchestrators, these are clustered/load balanced – often the first one will pick up work, but don’t rely on that – it could be any of them which picks up tasks – and it could be more than 1 for a given deployment run – for example server 1 picks up some of the tasks and server 2 picks up some other tasks – always check event logs on all of them to avoid missing anything useful.

To make it easier to check them you can add a custom view of the event logs on each orchestrator machine, to give you all the necessary logs in one place, like this:

I found in my case that server 2 was showing an error, as below, it’s basically saying it couldn’t find the AOS metadata service, and I notice the URL is wrong – I’d entered something wrong in the deployment settings in LCS:

AOS Machines

There are also useful logs on the AOS machines – the orchestrators are calling deployment scripts but for AX specific functions the AOSes are still running – for example database synchronize is run by an AOS. Again the AOSes are clustered so we need to check all of them as tasks could be executed by any of them. Similar to the orchestrators I create a custom event log view to show me all Dynamics related events in one place. This time I am selecting the Dynamics category, and I have unchecked “verbose” to reduce noise.

Here’s an example of a failure I had from a Retail deployment script which was trying to adjust a change tracking setting, for an issue such as this, once I know the error I can work around the problem by manually disabling change tracking on the problem table from SQL Server Management Studio and then starting the deployment again from LCS.

ADFS Machines

The ADFS servers will show authentication errors – typical causes of this kind of failure could be a “bad” setting entered in the deployment settings in LCS – for example I entered the DNS address for the ax instance incorrectly, then I see an ADFS error after deployment when trying to log into AX:

If you see an error as above, you can understand more about it my reviewing the Application group setup in “ADFS Management” on the ADFS machine, open it from server manager:

Under application groups you’ll see one for D365, double click it to see the details

If you’re familiar with the cloud version of D365, then you’ll probably know that AAD requires application URLs to be configured against it to allow you to log in – in cloud the deployment process from LCS is doing this automatically, and you can see it if you review your AAD setup via the Azure portal. In the on-prem version, this ADFS management tool shows you the same kind of setup, also in on-prem the deployment process is creating these entries automatically for you. Click on one of the native applications listed and then the edit button you can see what’s been set up:

The authentication error I mentioned previously:
MSIS9224: Received invalid OAuth authorization request. The received 'redirect_uri' parameter is not a valid registered redirect URI for the client identifier: 'f06b0738-aa7a-4a50-a406-5c1e486c49be'. Received redirect_uri: 'https://dax7sqlaodc1.saonprem.com/namespaces/AXSF/'.

We can now see from the configuration above that for client 'f06b0738-aa7a-4a50-a406-5c1e486c49be' the request URL isn’t configured. If we believed that the URL is correct, then we could add it here and ADFS would then allow the request to go through successfully. IN my case the URL was the mistake, so I didn’t change ADFS settings, I corrected the URL in LCS and started the deployment again.

Package deployment failures

When reconfiguring an environment, and including a custom package, if the deployment fails, check the orchestrator machine event logs, as described above – use a custom event log view to check all the logs on a machine at once.

I have had a situation where I’m getting failures related to package dependencies where my package does not have the failing dependency, I will explain:
Error is:

Package [dynamicsax-demodatasuite.7.0.4679.35176.nupkg has missing dependencies: [dynamicsax-applicationfoundationformadaptor;dynamicsax-applicationplatformformadaptor;dynamicsax-applicationsuiteformadaptor]]

My package does not contain demodatasuite, so the error is a mystery. Turns out that because my package has the same filename as a previously deployed package, the system is not downloading my package and just attempting to deploy an old package with the same name. Packages can be found in the file share location, as below:
\\DAX7SQLAOFILE1\SQLFileShare\assets

The first part, \\DAX7SQLAOFILE1\SQLFileShare, is my file share (so will differ in different environments – it’s a setting given when the environment was created), the assets folder is constant.

In here I see that my current package “a.zip” (renamed to a short name to work around an issue with deployment failure due to path too long), is from several weeks ago and is much larger than the package I expect. To get past this I rename my package to b.zip and attempt deployment again. Note that after PU12 for on-premise this issue no longer occurs.

Package deployment process

During the package deployment process, the combined packages folders will be created in this folder:

\\DAX7SQLAOFILE1\SQLFileShare\wp\Prod\StandaloneSetup-109956\tmp\Packages

Error when environment left in Configuration mode

When running a reployment, the error below can occur if the environment has been left in Configuration mode (for changing config keys), turn off configuration mode, restart the AOSes and then re-run the deployment.

MachineName SQLAOSF1ORCH2
EnvironmentId c91bafd5-ac0b-43dd-bd5f-1dce190d9d49
SetupModuleName FinancialReporting
Component Microsoft.Dynamics.Performance.Deployment.Commands.AX.AddAXDatabaseChangeTracking
Message An unexpected error occurred while querying the Metadata service. Check that all credentials are correct. See the deployment log for details.
Detail Microsoft.Dynamics.Performance.Deployment.Common.DeploymentException: An unexpected error occurred while querying the Metadata service. Check that all credentials are correct. See the deployment log for details. ---> System.ServiceModel.FaultException: Internal Server Error Server stack trace: at System.ServiceModel.Channels.ServiceChannel.HandleReply(ProxyOperationRuntime operation, ProxyRpc& rpc) at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs, TimeSpan timeout) at

Error when FRServiceUser is missing

This error can also happen when the FRServiceUser is missing in USERINFO – the AOS metadata service is trying to create an AX session as this user.
This user is normally created by the DB synch process. If the user is incorrect in USERINFO then deleting that User and re-running DB synch should recreate the user – you can set USERINFO.ISMICROSOFTACCOUNT to 0 in SSMS, and then re-run db synch to create the user. DB synch can be triggered in PU12+ by clearing the SF.SYNCLOG table and then killing AXService.exe – when it automatically starts back up it will run a db synch. The you should see the FRServiceUser created back in USERINFO.

MachineName SQLAOSF1ORCH2
EnvironmentId c91bafd5-ac0b-43dd-bd5f-1dce190d9d49
SetupModuleName FinancialReporting
Component Microsoft.Dynamics.Performance.Deployment.Commands.AX.AddAXDatabaseChangeTracking
Message An unexpected error occurred while querying the Metadata service. Check that all credentials are correct. See the deployment log for details.
Detail Microsoft.Dynamics.Performance.Deployment.Common.DeploymentException: An unexpected error occurred while querying the Metadata service. Check that all credentials are correct. See the deployment log for details. ---> System.ServiceModel.FaultException: Internal Server Error Server stack trace: at System.ServiceModel.Channels.ServiceChannel.HandleReply(ProxyOperationRuntime operation, ProxyRpc& rpc) at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs, TimeSpan timeout) at

Summit EMEA is about to happen, but we don’t have you on the list of attendees yet! Join nearly 1,500 Dynamics users and partners from all over Europe at the Convention Centre in Dublin from 24-26 April. At the moment there are still some seats left, claim yours today before they run out!

Register Now

Some Fast Facts

Here are some fast facts that will show why Summit EMEA is a can’t miss event. Did you know that….

….Summit EMEA offers 81 D365UG/AXUG sessions and 72 D365UG/CRMUG sessions across 16 tracks

….12 Academy classes are offered within Summit EMEA’s Pre-Conference Academy

….Summit EMEA also offers an Executive Programme specifically designed for Executive and Higher Management attendees

….61 exhibitors can be found at the Expo Hall

…users who are not yet a member of the User Groups receive a complementary one-year membership with their Summit EMEA registration

Before It’s Too Late

Summit EMEA is packed with opportunities to help you learn and be your best. Register now and join your Dynamics 365, AX & CRM user group community 24-26 April 2018 in Dublin! 

Register Now

Attend Focus 2018 to grow professionally, network with other Microsoft Dynamics 365/AX users and experts and most importantly, learn how to optimize your use of Dynamics 365/AX through advanced, deep-dive education. Don't miss out - Register now.

D365UG/AXUG Member Daniel Walter is a Business Analyst at Australian Gold and one of the Indianapolis Chapter Leaders for D365UG/AXUG. Read his welcome message below.

Hello,

I have been a part of the AXUG Indianapolis Chapter ever since I first started at my company 11 years ago. Back then we were on AX 2009 but have since moved to AX 2012 r3 and never looked back. 

As a company we benefit a great deal by participating in webinars, attending local meetings to network with our peers and attending larger conventions such as Summit and Focus. Being in IT and answering help desk tickets one of our first go to places to look for solutions to AX problems is the Open Forum on AXUG.com.

Events like Focus and Summit are wonderful opportunities to meet others and share common issues. Make the most of your time by networking with other users so after the conference ends you have a group of people to bounce ideas off and go to when issues come up. 

Regards,
Daniel

Enjoy what our host city has to offer

A lineup of only-in-Indy experiences make it a must-visit destination - Condé Nast Traveler even named Indianapolis one of their "Hottest Destinations of 2018". Here's why Indianapolis is an exciting and fitting location for this year's Focus host city:

• Dining: Indianapolis offers over 300 diverse dining options for all palates and price points. 
• Nightlife: When the work day is over, Indianapolis comes alive with hundreds of places for good times, good drinks, and good conversation. From happy hour hotspots to late night live music venues, you will find yourself in an activated downtown with endless entertainment and networking options conveniently close by.
• Transportation: Known as the Crossroads of America, Indy is within a day’s drive of over half of the country’s population. 

Time to get registered

As a bonus, attend Dynamics 365/CRM sessions happening concurrently at no additional charge. Take advantage of this can't-miss training opportunity and join fellow peers and technical experts in Indianapolis!

Register for Focus

Not yet a D365UG/AXUG member? Learn more about how user group members save $200 more than non-members on their Focus registration.

Availability:

The 2018-R3a Payroll tax update for Microsoft Dynamics 365 for Operations has been released. The update is available via LCS.

https://lcs.dynamics.com/Logon/Index

Release notes:

Please see the release notes below for details on the changes that were included in this tax update.

(Please visit the site to view this file)

After installing a hotfix (Ex, KB4090327) for Retail Return Location customers have been having issues with Return Locations not populating.

First, go to Retail > Inventory management > Return locations to create your Return location.

In my example if you select the Defect Subcode (1), the Returned item should go into the Central\Houston\Defected location (only if the item is tracking Locations). The Block inventory check box means that you are not able to sell out of the Defected Location.

The most common reason for the Return Locations not populating is:

• The Return Location is not added to the Retail Product Hierarchy.

Go to Retail > Product and categories > Product categories.  Select a node that has your product and expand the Manage inventory category properties fast tab.  Select your company and set the “Return location”

Please note that there are two ways to get into product categories (through hierarchy setup in products menu and through Retail menu) when you go in through product setup – it doesn’t show the inventory configuration section which needs to be used for this, so users must open the form from retail menus.  I do agree that this is very confusing, and this is a big reason why this step is missed and reported as a bug (when this is currently not considered a bug).

• The Site\Warehouse\Location is not Active
• Verify a customization is not preventing the Return location from populating.