Hi @lisbet-alvarez
I’ve tested it myself before, boundary search works fast without sudo
Obviously there is no problem in server API response time or cli, performance degradation seems to be GUI only problem
Also I’ve tested that it is not specific to my machine, by asking teammate to test GUI performance. He has the same issues
Thanks for the info.
We understand and to narrow this down more we just need some more numbers to see the difference. In one terminal can you run boundary cache status in order to retrieve the AuthToken Id and then run:
time curl -X GET --unix-socket /Users/roshin/.boundary/socket/daemon.sock "http://internal.boundary.local/v1/search?resource=targets&force_refresh=true&auth_token_id=<TOKEN_ID>" > /dev/null
Please repeat a couple of times and send us screenshots of the output.
In a separate terminal, please run the desktop client with logging enabled using BOUNDARY_DESKTOP_LOG_LEVEL=debug ./Boundary.app/Contents/MacOS/Boundary, and the logs are located at ~/Library/Logs/Boundary/desktop-client.log. Please search targets a couple of times and send us the logs of that as well.
You can use boundary authenticate to log in through the CLI and then boundary cache status should have user information. Apologies this was a missing step from my comment above.
@lisbet-alvarez here is the requested info:
[2024-08-12T20:12:58.950+03:00] [info] Boundary Desktop Client Version: 2.1.0 | Commit: fc5121c41e5d6387785e70e8a396f23d24f073ae
[2024-08-12T20:13:00.390+03:00] [info] Boundary CLI Version: 0.17.0 | Commit: af0e89c328b9acf37925df791a85930418792cf7
[2024-08-12T20:13:03.854+03:00] [info] Cache daemon started, status from daemon:
The cache is already running.
[2024-08-12T20:13:32.297+03:00] [debug] Search request took 496 ms {
query: '(user_id = "u_Qifo7VLXaA") and (status = "active" or status = "pending")',
force_refresh: true,
filter: '"/item/scope/parent_scope_id" == "o_lZWMJEQLsp"',
resource: 'sessions'
}
[2024-08-12T20:13:33.718+03:00] [debug] Search request took 1931 ms { force_refresh: true, query: '', resource: 'resolvable-aliases' }
[2024-08-12T20:13:33.722+03:00] [debug] Search request took 1923 ms {
force_refresh: true,
filter: '"/item/scope/parent_scope_id" == "o_lZWMJEQLsp"',
query: '',
resource: 'targets'
}
[2024-08-12T20:13:34.183+03:00] [debug] Search request took 465 ms {
query: '',
force_refresh: true,
filter: '"/item/scope/parent_scope_id" == "o_lZWMJEQLsp"',
resource: 'targets'
}
[2024-08-12T20:15:18.900+03:00] [debug] Search request took 3061 ms {
query: '(user_id = "u_Qifo7VLXaA") and (status = "active" or status = "pending")',
force_refresh: true,
filter: '"/item/scope/parent_scope_id" == "o_lZWMJEQLsp"',
resource: 'sessions'
}
[2024-08-12T20:15:18.904+03:00] [debug] Search request took 3067 ms { force_refresh: true, query: '', resource: 'resolvable-aliases' }
[2024-08-12T20:15:20.230+03:00] [debug] Search request took 1322 ms {
query: '(user_id = "u_Qifo7VLXaA") and (status = "active" or status = "pending")',
force_refresh: true,
filter: '"/item/scope/parent_scope_id" == "o_lZWMJEQLsp"',
resource: 'sessions'
}
[2024-08-12T20:15:20.231+03:00] [debug] Search request took 1323 ms { force_refresh: true, query: '', resource: 'resolvable-aliases' }
[2024-08-12T20:15:21.592+03:00] [debug] Search request took 1361 ms {
query: '(id % "new" or name % "new" or description % "new" or address % "new" or scope_id % "new")',
force_refresh: true,
filter: '"/item/scope/parent_scope_id" == "o_lZWMJEQLsp"',
resource: 'targets'
}
[2024-08-12T20:15:22.958+03:00] [debug] Search request took 1362 ms {
query: '',
force_refresh: true,
filter: '"/item/scope/parent_scope_id" == "o_lZWMJEQLsp"',
resource: 'targets'
}
[2024-08-12T20:15:31.241+03:00] [debug] Search request took 2994 ms {
query: '(user_id = "u_Qifo7VLXaA") and (status = "active" or status = "pending")',
force_refresh: true,
filter: '"/item/scope/parent_scope_id" == "o_lZWMJEQLsp"',
resource: 'sessions'
}
[2024-08-12T20:15:31.244+03:00] [debug] Search request took 2998 ms { force_refresh: true, query: '', resource: 'resolvable-aliases' }
[2024-08-12T20:15:32.597+03:00] [debug] Search request took 1352 ms {
query: '(user_id = "u_Qifo7VLXaA") and (status = "active" or status = "pending")',
force_refresh: true,
filter: '"/item/scope/parent_scope_id" == "o_lZWMJEQLsp"',
resource: 'sessions'
}
[2024-08-12T20:15:32.597+03:00] [debug] Search request took 1352 ms { force_refresh: true, query: '', resource: 'resolvable-aliases' }
[2024-08-12T20:15:33.936+03:00] [debug] Search request took 1338 ms {
query: '(id % "n" or name % "n" or description % "n" or address % "n" or scope_id % "n")',
force_refresh: true,
filter: '"/item/scope/parent_scope_id" == "o_lZWMJEQLsp"',
resource: 'targets'
}
[2024-08-12T20:15:35.268+03:00] [debug] Search request took 1334 ms {
query: '(id % "new" or name % "new" or description % "new" or address % "new" or scope_id % "new")',
force_refresh: true,
filter: '"/item/scope/parent_scope_id" == "o_lZWMJEQLsp"',
resource: 'targets'
}
[2024-08-12T20:15:48.199+03:00] [debug] Search request took 2926 ms {
query: '(user_id = "u_Qifo7VLXaA") and (status = "active" or status = "pending")',
force_refresh: true,
filter: '"/item/scope/parent_scope_id" == "o_lZWMJEQLsp"',
resource: 'sessions'
}
[2024-08-12T20:15:48.200+03:00] [debug] Search request took 2928 ms { force_refresh: true, query: '', resource: 'resolvable-aliases' }
[2024-08-12T20:15:49.616+03:00] [debug] Search request took 1411 ms {
query: '(user_id = "u_Qifo7VLXaA") and (status = "active" or status = "pending")',
force_refresh: true,
filter: '"/item/scope/parent_scope_id" == "o_lZWMJEQLsp"',
resource: 'sessions'
}
[2024-08-12T20:15:49.617+03:00] [debug] Search request took 1412 ms { force_refresh: true, query: '', resource: 'resolvable-aliases' }
[2024-08-12T20:15:50.980+03:00] [debug] Search request took 1362 ms {
query: '(id % "re" or name % "re" or description % "re" or address % "re" or scope_id % "re")',
force_refresh: true,
filter: '"/item/scope/parent_scope_id" == "o_lZWMJEQLsp"',
resource: 'targets'
}
[2024-08-12T20:15:52.338+03:00] [debug] Search request took 1359 ms {
query: '(id % "redis" or name % "redis" or description % "redis" or address % "redis" or scope_id % "redis")',
force_refresh: true,
filter: '"/item/scope/parent_scope_id" == "o_lZWMJEQLsp"',
resource: 'targets'
}
Hey @lisbet-alvarez any results on analyzing the debug?
Hi @dmitryroshchin,
Thank you for providing logs. We are still looking into this issue.
I am curious that the queries for resolvable-aliases & sessions (when we load/search targets) are strangely taking longer than the query for targets. Is this because you have many aliases pointing to targets and existing sessions? I am mainly asking to make sure there aren’t other underlying issues.
@lisbet-alvarez I don’t have aliases at all, because my terraform code was initially created on version 0.14 which didn’t support aliases. Maybe I should have mentioned it before but, for current gui version to work I had to upgrade controllers and workers from 0.14 to 0.16
But I don’t think it matters because server side APi works fast
And I have no more than 5 simultaneous sessions at the moment
Also listing sessions from cli works really fast:
boundary sessions list --recursive 0.04s user 0.04s system 4% cpu 1.784 total
@lisbet-alvarez please give me hope for successful resolution:pray:
Thank you for your patience. After some internal conversation with the team, we suggest you to update Boundary controller and workers to the latest version, as per today is 0.17.1.
We also recommend using the latest version of the desktop client, as per today 2.1.0.
We will appreciate after the upgrades you report back with the results.
Thank you
Hi @calcaide , sorry for the delayed answer
I’ve upgraded boundary to 0.18.0 and desktop client to 2.2.0
Nothing changed still slow response without sudo, and almost immediate when running with sudo
Obviously cache daemon only works properly when run as root
@calcaide @lisbet-alvarez hello!
Really clould use your help in solving this issue, don’t know where to dig further
Please assist!
Sorry for my late response. We have not been able to reproduce the described issues.
hello @calcaide
New update on the issue, I hope it will help:
I’ve just tested boundary desktop on my old amd64 system and it works fast as it should
No performance degradation at all!
On both systems OS is MacOS 15.1 and Boundary Desktop Client Version: 2.2.0
I guess this proves that there is no configurations issues on controller’s side and It’s strange that you didn’t meet the same issues with other installations
The issue seems to be existing only with arm version of the desktop and/or cache daemon. What other debugging steps could we do to pinpoint it?
Thanks for reporting back. The extra information definitely helps.
Agreed with you, this proves the issue is within the Mac ARM machine, you are reproducing issues.
We performed different tests to attempt to reproduce your issues without success, we use the Mac ARM daily, it’s unfortunate, and we are sorry about that.
I know we already discuss it within this thread, but just to validate you are running the ARM version of the DC, do you mind checking the Mac OS Activity Monitor? With Boundary DC running, open the activity monitor, click on CPU tab, search (top-right margin) for Boundary process, within the column “Kind” of the processes list, Boundary should be “Apple”. That will confirm you are running the ARM version.
Looking for other possible issues here, within the ARM machine, do you have some sort of software that interacts with DNS? If is an enterprise machine provided by an employer, it might have some security software?
As per your question on debugging steps. The desktop client has logs, in those logs you can find how long each request took. In case you want to compare ARM vs AMD64 performance. Within the desktop client, navigate to settings, scroll down to “Logs” it will show you where the log file persist in your system.
I hope this helps
Problem solved, the reason was in our threat prevention module. After adding boundary DC to whitelist it works fast as it should.
I should have checked it much earlier!
@calcaide thanks a lot for the hint on security software!
One more question
Although boundary DC works fast now, there is still that error of cache, what could be the reason of it?
I am glad the issue got sorted out.
About the security software, if you don’t mind, we will appreciate it if you can share with use what security software you are using? So we can document it for future reference.
About cache daemon issue:
For what we can see in the screenshot, we think it can be related to the grants the user has assigned. Perhaps those grants were created under 0.16 or 0.17 controller? before the introduction of aliases, I will review those. Also, when doing grant changes on the controller, you will need to log out/log in within the DC for those to be refreshed.
Hi @calcaide the security software is jamf protect but I believe similar issues would be caused by other security products too