In February 2023, we completed an internal analysis to explore what AI could do in SAP consulting, using ChatGPT as an example. We later published some of our findings in the blog post “ChatGPT vs. SAP Consultant: Will artificial intelligence make SAP consultancies obsolete?”
Admittedly, the article now feels almost completely outdated given the speed of development in generative AI.
Our conclusion at the time was that AI can indeed be helpful for getting up to speed on new subject areas but still showed clear shortcomings when it came to specific SAP knowledge. One finding, however, still holds true today: AI-generated results must always be critically reviewed.
Why AI Results Still Need Critical Review
It isn’t just the capability of the AI models themselves that has increased; the tooling surrounding these models has also advanced considerably. AI systems are now increasingly agentic. Developments such as Retrieval-Augmented Generation (RAG) combined with intelligent web search, along with new standards like the Model Context Protocol (MCP), Agent-to-Agent (A2A), and specialized skills, are continuously reducing incorrect answers (hallucinations).
To properly assess AI-generated results, humans now need considerably deeper subject-matter expertise. When hallucinations do occur, they are often phrased so convincingly and even appear logically coherent within the given assumptions that AI-generated errors are recognized much later. This creates the risk that assumptions based on such errors, for example when planning projects, initially appear plausible but later turn out to be false and unrealistic.
An example from a tender
In practice, we have already reviewed tenders that showed clear signs of generative AI having been used. In one case, the planning was based on the assumption that certain customizing settings in an SAP module were not relevant. Accordingly, no effort was budgeted for them. Our experts had to correct this assumption, which also changed the assumed project scope.
How AI Is Changing SAP Consulting Today
Roles involving repetitive work and little deep expertise are at risk, even in SAP consulting. What matters is achieving higher productivity through the correct use of AI. What is not enough: simply copy-pasting tasks through AI.
Example: The modern Project Management Office (PMO)
Modern PMOs no longer simply take notes; they have meetings transcribed by AI. They set up meetings so that the relevant AI functions run from the very start, for example with the correct language configured. The resulting minutes are then filed in a structured way via a workflow and subsequently reviewed critically. During the meeting itself, the PMO increasingly takes on the role of a moderator. Finding suitable dates and available time slots can likewise be supported by AI. Onboarding is also no longer handled manually: welcome emails, permissions, appointment forwarding, and other onboarding tasks are implemented through automated workflows.
Undoubtedly, correctly applied AI reduces the personnel effort required for certain tasks. However, the PMOs that remain needed can, with AI support, handle a larger volume of work and correspondingly take on more responsibility. The complexity of a modern PMO should not be underestimated: familiarity with various AI and automation tools makes the work technically more demanding as well. If, for example, a workflow fails, a PMO must be able to fix the error itself.
Customers are becoming more capable too
Another factor must not be overlooked: customers are using AI and automation themselves. Customers that use AI effectively can perform many typical tasks of senior SAP consultants largely on their own and may only need targeted support. When they do, however, they expect absolute expert-level capability.
SAP consultancies that lack a broad base of experts and do not pursue continuous AI enablement for their staff will increasingly fall behind in the market. What we already do at adesso today, and what we also see in the market, is that efficiency gains from AI are made transparent to customers and shared with them.
New Practical Test: 17 SAP Expert Questions, Four AI Systems
For our new test, we used four different AI system configurations:
- Google Gemini: Gemini 3.6 Thinking with web search enabled
- SAP Joule for Consultants: J4C
- Anthropic Claude: Claude Opus 5 – High, with web search enabled, an adesso skill “SAP PSM Consultant,” and active MCP access to SAP Help search
- adessoGPT: OpenAI GPT 5.6 Sol Thinking, without web search but connected to Microsoft Graph and the intranet
The 17 questions came from the area of SAP Public Sector Management (PSM). Compared with the 2023 test, we used significantly more questions this time, and the questions were considerably more complex, operating at SAP expert level. Unlike in 2023, we did not submit each question in a separate prompt – instead, we submitted all 17 questions together in a single prompt.
The range of questions spanned from general questions that can be answered directly by searching known SAP documentation, to questions requiring an understanding of the SAP component (e.g., question 2 / use of financial transactions).
An overview of the results
| Question | Google Gemini | SAP Joule for Consultants (J4C) | Anthropic Claude | adessoGPT |
| Which attribute of a commitment item is critical for the correct posting of documents to Funds Management? | Correct | Correct | Correct | Correct |
| Which value type and amount type are updated in the budget when two commitment items are posted with financial transaction 30? | False | False | False | False |
| Which master data must be used when implementing Funds Management (PSM-FM)? | Correct | Correct | Correct | Correct |
| What is Payac? | False | False | False | False |
| What is the difference between grantor and grantee? | Correct | Correct | Correct | Correct |
| What master data is required to use SAP Grants Management (Grantee)? | False | False | Correct | False |
| What is CVI and is it required for SAP Grants Management (Grantee)? | Correct | Correct | Correct | Correct |
| Is there a budget integration between SAP Funds Management and Grants Management (Grantee)? | Correct | Correct | False | Correct |
| Can I use classic budgeting with SAP Grants Management (Grantee)? | Correct | False | Correct | Correct |
| What are the main changes in SAP S/4HANA for Funds Management and Grants Management (Grantee)? | False | Correct | Correct | Correct |
| What are the relevant Customizing steps for setting up the Budget Control System (BCS)? | False | False | Correct | Correct |
| Are there budget subtypes in the Budget Control System? | False | False | Correct | Correct |
| In which main tables are budget values updated in BCS? | False | False | Correct | False |
| How can I migrate from classic budgeting to BCS? | Correct | Correct | Correct | Correct |
| What impact does the new Asset Accounting have on SAP Funds Management? | False | False | False | Correct |
| Via which Customizing settings can I deactivate Funds Management? | Correct | Correct | Correct | Correct |
| What are prohibited business transactions in Funds Management? | False | False | False | False |
Depending on the configuration, eight or twelve questions were answered correctly:
| Google Gemini | SAP Joule for Consultants | Anthropic Claude | adessoGPT |
| 8 | 8 | 12 | 12 |
Was die Ergebnisse zeigen
SAP Joule for Consultants and Gemini each answered 8 of 17 questions correctly. This is notable because SAP J4C is specifically marketed as being built on SAP expertise, whereas Google Gemini is a general-purpose AI model that achieved the same score in this test.
The costs of SAP J4C depend on the negotiated price per AI Unit. By default, 35 AI Units per user per month are required, amounting to €245 per month based on the pricing used for this comparison. Google Gemini may be less expensive, depending on their licensing and usage models.
Claude and adessoGPT achieved the best results, each with twelve correct answers.
For adessoGPT, our internal calculation put the cost for all questions combined at around €2. Alternatively, a different model operated by adesso could have been used instead; however, whether that would likewise have yielded twelve out of 17 correct answers would need to be verified. In any case, this variant would also have offered the advantage of access to internal information sources.
In the tested configuration, Anthropic Claude incurred monthly costs of €50 per user. Further investigations could examine how disabling MCP access or the skill, or varying the model used, affects the results.
Overall, it should be noted that all the systems tested showed impressive knowledge in the area of SAP PSM. What was particularly surprising was that SAP J4C did not perform better in this test — in some cases it even performed worse than other AI systems. In addition, the chosen price point for SAP J4C appears too high relative to its performance.
The conclusion must be viewed in light of the defined framework conditions. The test looked at the clearly delimited subject area of SAP PSM, and it represents a snapshot in time.
How adesso business consulting Uses AI and Automation
Responsible use of AI is firmly embedded organizationally at adesso. This includes, in particular, quality assurance of AI results, the use of sovereign AI models, and transparent handling of productivity gains, including sharing them with our customers.
We have integrated the consistent use of optimization potential into our project management methodology. This enables us to identify not only which AI and automation solutions from the SAP ecosystem are relevant to customers, but also which existing non-SAP AI tools can be used and connected to SAP through MCP or A2A. A more detailed article is available at “AI and automation in SAP transformation projects”.
Conclusion
We need to revise some of our findings from 2023. SAP consultancies that do not invest substantially in both initial and continuous AI enablement for their employees risk being left behind.
AI models have become far more capable, and the quality of answers involving SAP expert knowledge is impressive. This progress is not due to the models alone; the surrounding tooling also plays a major role. RAG, including web search, MCP, A2A, and skills are among the most prominent examples.
In our view, the risk of obviously incorrect results has clearly decreased as a result. However, it still exists and has become more dangerous, since errors are now recognized much later.
Even though SAP J4C and Gemini achieved good results in our test, Claude and adessoGPT were the most convincing configurations. However, the result is a snapshot in time and can change quickly. Customers should therefore check what possibilities their existing tools already offer, and whether additional AI licenses are actually required for a given use case.
Compared with 2023, something else has changed fundamentally: thanks to the agentic capabilities of AI tools and the ability to connect them directly to SAP systems, customizing analyses, programming tasks, and other activities can now be carried out directly in conjunction with productive SAP systems.
This does not mean SAP consultants are becoming fundamentally obsolete. However, their tasks, role profiles, and the demands on their professional and technical skills are changing fundamentally.
