Project Talk:Infrastructure/Shopping list

Hi Needle,

The financial statements are public: Foundation:Gentoo_Foundation_Finances_FY2021 Neither switch is a problem for the budget.

This is going into the single rack at OSL that has the Gentoo systems. It has least two uplink ports today, maybe more, on different L2 networks.

We do have a number of 100M/1G links in the rack as well, that I didn't document, e.g. OOB/IPMI ports, or embedded systems, probably worthwhile to better capture them for planning.

The power budget is probably the next most important thing I should look at, on that side and I'm astounded by how low-power (85W) the S5860-20SQ is, with both of the other options (S5860-24XB-U, S5860-48SC) being 550W/1KW. It seems almost too low-power.

The lowest power solution would seem to be something like the S5860-20SQ plus a separate very cheap managed 1G switch for OOB. e.g. 24W for S3910-24TF https://www.fs.com/products/115382.html.

--Robbat2 (talk) 22:13, 25 November 2021 (UTC)

Heya Robbat2,

your option sounds good. My proposition was based only on the requirements written down on the shopping list.

Since you know more about the gentoo budget available for that networking gear and the probable future use of ports and how many would be needed, also the local ports available in the datacenter where the gear is placed in the univeristy.

Under the hood there is not much difference between the switches, apart from bandwidth/optics/amount of ports. Regarding 40G ports moving to 100G ports, YES. 40G optics are going away and are more complex than 100G optics. Both models are cool. I did not suggest any 25G optics since it is not listed in the requirements and these need to match your future server gear. 10G/100G is mainstream and there will be no problems with that.

Only thing regarding breakout cables, these tend to fail sometimes, and if you lose 1 breakout cable then in worst case 4 ports may fail, also replacing breakout cables and troubleshooting them is hassle, wanted to spare you this trouble. Personally I do not like breakouts because of complexity and material failing. Some breakouts are heavy and the cable's are hanging, temperature and surroundigs, rack doors and mechanical action make the material fail if it has bad quality. Sometime after few years you only need to touch them while patching and you get problem with CRC errors. I had bad experience with breakouts, reason I am not using them. But this can be different in your case.

Just wanted to suggest gentoo does not buy some overpriced networking gear form big networking companies, reason for suggesting FS networking gear for gentoo. Go for it. Needle (talk) 21:44, 25 November 2021 (UTC)

Hi Needle,

Thanks for the suggestion. One of the ideas I already had was this unit from FS: https://www.fs.com/products/108710.html S5860-20SQ, 24-Port Ethernet L3 Fully Managed Pro Switch, 20 x 10Gb SFP+, with 4 x 25Gb SFP28 and 2 x 40Gb QSFP+

And then explicitly purchasing enough 10GBASE-T SFP+ transceivers to use with it. https://www.fs.com/products/74680.html

The S5860-20SQ switch is only $1600USD, vs the $3800USD for S5860-24XB-U. Even if were fully populated w/ the transceivers, that's 20x$65USD = extra $1300

Puts the upper bound at $2900USD, and gives other options (e.g. put some of the newest hosts on 25G for VM replication, or use the QSFP+ ports for future expansion)

The logical expansion of that idea could be: https://www.fs.com/products/115385.html?attribute=5271&id=205746 S5860-48SC, 48-Port Ethernet L3 Fully Managed Pro Switch, 48 x 10Gb SFP+, with 8 x 100Gb QSFP28 which has a base price of $4K USD. Could get 4x25G by using $500USD breakouts, but wouldn't get any 40G options at all (probably not a problem since 40G is being replaced by 100G)

--Robbat2 (talk) 19:41, 25 November 2021 (UTC)

Hi for switches with 10GBase-T you could check these both, depending on how much ports for future use you need:

24 Port variant: https://www.fs.com/products/108716.html

48 Port variant: https://www.fs.com/products/69378.html

BCM chipset, having all the stuff you would need, including TACACS/RADIUS authentication, LACP, CLI access, redundant PS. Much like the cisco switch you would replace. Both have 10GBase-T. The 24 Port variant has option for additional 4x SFP+; the 48 Port variant has only copper 10GBase-T ports, no SFP+, 4x 40G uplinks with opiton for breakout-cable 4x10G . I'd recommend the 24 port variant, looks to me like the 48 port is more expencive and no easy way for SFP+ (breakout cables from 40G to 4x10G tend to fail)Needle (talk) 22:26, 24 November 2021 (UTC)