Repository navigation
Azure SDK is over 500MB and growing on each release. #17801
Description
The azure SDK is ridiculously large for reasons that I have a hard time understanding. We pip install it for our CI pipelines and the vast majority of the size of our container is coming from the Azure SDK, in the SDK the network directory is taking almost half of the size and this is because there are 39 versions of the SDK.
I have never seen anyone doing such a strange approach to version their API clients. I fail to understand why anyone would even want to use the client from 2015 on a cloud product like Azure.
root@1bba10bd1500:~/.pyenv/versions/3.9.2/lib/python3.9/site-packages/azure/mgmt/network# du -shc * | grep M | sort -n
1.2M aio
2.4M v2015_06_15
3.3M v2016_09_01
3.5M v2016_12_01
3.7M v2017_03_01
4.4M v2017_06_01
4.4M v2017_08_01
4.9M v2017_09_01
5.1M v2017_10_01
5.1M v2017_11_01
5.1M v2018_01_01
5.7M v2018_02_01
6.5M v2018_04_01
6.6M v2018_06_01
6.9M v2018_07_01
8.3M v2018_08_01
8.4M v2018_10_01
8.6M v2018_11_01
8.8M v2018_12_01
9.0M v2019_02_01
9.5M v2019_04_01
10M v2019_06_01
11M v2019_07_01
11M v2019_08_01
11M v2019_09_01
11M v2019_11_01
11M v2019_12_01
12M v2020_03_01
12M v2020_04_01
13M v2020_05_01
13M v2020_06_01
13M v2020_07_01
13M v2020_08_01
259M total
Can the default release only prove the latest version of the client libraries, or at least provide a 'lean' version of the SDK? This release model is certainly not sustainable and is causing useless grief to your users.
Reacted by Saurav Sarkar, Anthony Cruz, amberclumio, viveksk-clumio, Michael J. Schultz, Jiashuo Li, Lenon, Robert Oliveira, Travis Cunningham, James and 18 more
Activity
kristapratico commentedon Apr 5, 2021
Hi Stephane Odul (@sodul), thanks for the feedback, we'll investigate asap.
jiasli commentedon Jun 22, 2021
Previously reported in #11149.
sodul commentedon Jun 23, 2021
To clarify #11149 is only about azure-mgmt-network which is the largest directory but the problem is present across the entire Azure SDK.
I understand the reasoning for the approach to keep everything for backward compatibility but if you do have customers that point to the old versions then they should pin their requirement versions to the old pypi.org releases of the Azure SDK, not force everyone to keep a copy of everything around. How about providing two versions of the SDKs: one large with everything, one small with just the latest version.
nolaexe commentedon Aug 5, 2021
Hey, is there any update?
sodul commentedon Aug 5, 2021
I wrote a script that we run after
pip install. It detects the unused versions and this got us an azure folder shrink from ~ 680MB to ~ 280MB. It cannot go any lower because for some reason some of the objects model definitions from multiple versions are merged together to make the final list that is then used. The script detects the versions that are used internally by the SDK and preserves them, making the script very safe to use.If there is interest I can open source the script.
sodul commentedon Aug 16, 2021
We have released our script on GitHub. It does delete a good chunk of the API folders but not all of it. With the script the Azure directory is now just under 300MB instead of over 700MB. It is compatible with most, but not all, third party packages, as long as they do not point to a version that is trimmed.
https://github.com/clumio-code/azure-sdk-trim
KranthiPakala-MSFT commentedon Nov 1, 2021
Krista Pratico (@kristapratico) Following up to see if there is any update on this issue? - Thank you
79 remaining items
iscai-msft commentedon Oct 11, 2024
i agree, and I apologize again for all of the issues you guys have been dealing with. I'm going to reach out more to the folks working on this. Really sorry for this still being a continued issue.
Lawouach commentedon Oct 11, 2024
I think your commitment to move the needle in the right direction is great.
lmazuel commentedon Oct 11, 2024
Thanks for commenting on this thread, this is highly appreciated. We're 100% aware of this, and we tried different approach to solve this problem (you may have noticed that azure-mgmt-network is smaller than it used to be), but there are unfortunately a few tooling (ironically this includes the CLI), that are designed assuming they can install multi-api SDK and load API-version by module name. I hope we can reach a conclusion that solve this problem for all party involved soon, I'll keep you guys updated, this issue is bookmarked in my top bucket list.
msyyc commentedon Nov 5, 2024
Status update:
SDK team has converted part of multiapi packages into singleapi packages. The size of those packages including network is reduced by 61%~94%: #38009
lmazuel commentedon Dec 10, 2024
Additional update: we have all agreed, including CLI team members, that SDK should stop shipping multi-api SDK entirely. A SDK will target the last known API version as the expected contract. It will still be possible to pass an
api_versionparameter, but the SDK will not verify if the parameter set passed is conformed to that API version and will let server signal any troubles with a server exception. It's recommended for people that rely on a very specific API version, to pin the SDK package to the version where that API version was the last one (so you get proper typing help at least).We're rolling this on all packages over the next couple of months.
Part of this effort, we will also split some packages into sub packages. For instance,
disksfrom theazure-mgmt-computepackage will have its own individual package. The rational is thatdisksnever versioned its service with the same API versions than regular compute operations (like VM), making that package complex to version, as it actually contained a combination of API versions of various resource types. The bad side effect, is that it will be a breaking change whendiskswill disappear fromazure-mgmt-compute, but the good side effect is even smaller SDK (less API versions and less resource types, both contributing) and a more consistent API version timeline (a unique one per package). The breaking change should be minimal (install new package, and create new client, everything else is still the same, no operation or model change).We will do this split on a few packages, the exact list is TBD, but if you're aware of a SDK with a mix of various API versions, he's probably on the future list.
lmazuel commentedon Oct 20, 2025
We have finally re-shipped every single packages of the SDK with simplified single API version, marking this issue as resolved,
Edit: Since initially this issue was about package size of Network, a few wheel size comparison
github-actions commentedon Oct 20, 2025
Hi Stephane Odul (@sodul). Thank you for opening this issue and giving us the opportunity to assist. We believe that this has been addressed. If you feel that further discussion is needed, please add a comment with the text "/unresolve" to remove the "issue-addressed" label and continue the conversation.
github-actions commentedon Oct 27, 2025
Hi Stephane Odul (@sodul), since you haven’t asked that we
/unresolvethe issue, we’ll close this out. If you believe further discussion is needed, please add a comment/unresolveto reopen the issue.