we are attempting a solution to replicate data(transactional as well as reference) from microsoft dynamics AX to MongoDB
after reading a lot of blogs and doing brainstorming among my team, i have arrived at following three solutions
1. Creating Document Services over any transfer protocol
poll through it and obtain the changes that have been done with regards to tables that document service is looking on
Fetch changed keys and subsequently obtain the row records based on changed keys
Pros:
since it is using AX document service, writing custom query or service with data source having multiple tables will make the process easier
Cons:
each call to document service at an interval of 30 to 60 seconds for 40 services will burden the system and impact performance
2. Enable CreatedDateTime and ModifiedDateTime system columns on tables inside AOT, write custom stored procedures which will run at certain intervals and retrieve rows which have changed or newly created after last retrieval.
Pros:
simpler approach and removes the dependency on dynamics ax
Cons:
we are unable to handle deleted records taking place in db with this approach
3. Turn on CDC on microsoft SQL Server residing behind Dynamics AX and monitor the change logs to replicate changes in mongo DB via message queues
Cons:
Enabling change tracking will slow down the performance of SQL Server and make it less responsive
is there any method through which we could get real time data push of table changes that a certain business flow has made?
could you suggest on best practices that are employed for replication using microsoft dynamics ax? also are there are any other approaches which we have not considered