ZFSSA 2 EXA


ZFS Storage Appliance Infiniband

To attach your ZFSSA using infiniband to an EXA for example you might want more than one "virtual" datalink on your HCA using multiple IB partitions with same pkey. It is (yet?) not possible to allow the use of the same pkey when creating two IB partitions (datalinks) that point to the same IB device using the BUI and the CLI property is hidden; don't know why; but I could find a workaround in: Oracle Support Document 2087231.1 (Guidelines When Using ZFS Storage in an Exadata Environment) https://support.oracle.com/epmos/faces/DocumentDisplay?id=2087231.1

In my example I will create a bunch of datalinks to enable an active-active IPMP failover configuration for both controllers. (you will have to start with ibpart1, "0" does not work)

zsexa0101a:configuration net datalinks>partition
zsexa0101a:configuration net datalinks partition (uncommitted)> set li    <-- tab tab
linkmode  links
zsexa0101a:configuration net datalinks partition (uncommitted)> set link  <-- tab tab
linkmode  links       <-- it is not there ;-| 
zsexa0101a:configuration net datalinks partition (uncommitted)> set linkname=ibpart5
                      linkname = ibpart5 (uncommitted)
zsexa0101a:configuration net datalinks partition (uncommitted)> show
Properties:
                         class = partition
                         label = Untitled Datalink
                         links = (unset)
                          pkey = (unset)
                      linkmode = cm
 
zsexa0101a:configuration net datalinks partition (uncommitted)> set links=ibp0
                         links = ibp0 (uncommitted)
zsexa0101a:configuration net datalinks partition (uncommitted)> set pkey=ffff
                          pkey = ffff (uncommitted)
zsexa0101a:configuration net datalinks partition (uncommitted)> show
Properties:
                         class = partition
                         label = Untitled Datalink
                         links = ibp0 (uncommitted)
                          pkey = ffff (uncommitted)
                      linkmode = cm
 
zsexa0101a:configuration net datalinks partition (uncommitted)> commit
zsexa0101a:configuration net datalinks> show
Datalinks:
 
DATALINK       CLASS       LINKS       STATE   ID      LABEL
aggr1          aggregation i40e0       up      -        zsexa01-LACP
                           i40e4
ibpart1        partition   ibp0        up      -        zsexa01-IB0
ibpart2        partition   ibp2        up      -        zsexa01-IB1
ibpart3        partition   ibp0        up      -        zsexa01-IB0
ibpart4        partition   ibp2        up      -        zsexa01-IB1
ibpart5        partition   ibp0        up      -       Untitled Datalink
igb0           device      igb0        up      -       Motherboard-igb0
pffff_ibp0     partition   ibp0        up      -        zsexa01-IB0
pffff_ibp2     partition   ibp2        up      -        zsexa01-IB1
vnic1          vnic        igb0        up      -        zsexa0101a-VNIC
vnic2          vnic        igb0        up      -        zsexa0102a-VNIC
vnic3          vnic        aggr1       up      -        zsexa0101c-VNIC
vnic4          vnic        aggr1       up      -        zsexa0102c-VNIC

SPARC performance still rulez!

Since Oracle JAVA like many other software is charged per core it might be more and more interesting where to get the most performance out per license… remember the announcement 3 years ago when Oracle brought their M8 SPARC chips – double JAVA performance than x86 and Power. Three years later that’s still true and proven by public SPEC benchmarks which show amazing results for M8 based servers. Compared to the latest and greatest x86 chips from Intel and AMD the latest SPARC chip still has 100% more capacity for jOPS and leads the critical jOPS per core! That would mean in a commercial point of view you would need double licenses for the same workload on other platforms.

And don’t forget the incredible performance when it comes down to In-Memory analytics. When M8 was announced the chip provided 10x faster results using the build on DAX engines than any other vendor. I saw comparisons now a days still showing 8x faster queries than the rest on market.

And in the end everyone talks about security but nearly no one encrypts their workload; most of the time because it ruins the critical performance. That’s not true on SPARC – encrypt all your business but only loose 2 to 4 % with end-to-end encryption and all performance features mentioned before.

So, with Moore’s law you could get the performance which will show up probably in 2 to 4 years. But looking back to the last 10 years of Intel’s single core performance it grew by 10 to 15%, especially when it comes down to multi core environments where Intel’s chips lose their ability to increase turbo modes on specific cores. On modern Platinum chips they scale very well up to 14-16 cores but using 24 or even more cores you are thrown back to the performance you got 10 years ago with 2GHz… it’s still true, if you need enterprise class systems with a predictable performance and linear scale you have to go with enterprise CPUs like SPARC.
(or Power – did I really say that?)

See the whole presentation from Bill Nesheim, SVP Oracle Solaris Engineering:
Oracle SPARC & Solaris Consistent, Simple, Secure

Oracle Hard Partitioning with Oracle Linux KVM

Just found the official note reading “Oracle Partitioning Policy” that Oracle KVM is now supported as Hard Partitioning (!).
“For sure” it is only supported with Oracle Linux KVM in a “special” documented setup:
Hard Partitioning Implementation with Oracle Linux KVM and Oracle Linux Virtualization Manager

A similar setup like it was on OVM with Xen – a management server called “Oracle Linux Virtualization Manager” with an Oracle script called “olvm-vmcontrol” to set the core pinning on KVM compute nodes.

As it seems it is still like the Xen environment free to use – but if you want support on 3rd party hardware you will have to buy Oracle premier support (not basic); but the KVM manager is listed as a premier supported feature – with OVM Xen you had to buy an own contract for OL and OVM.

These are good news because the Xen implementation will run out of support soon -> MAR2021 premier support for OVM3.x will end.
And as you could see around OOW19 all new products came out with KVM rather than Xen… Oracle Cloud is not running on Xen and EXA-X8M & ODA-X8 where announced running KVM virtualization (only PCA X8 on-prem lags behind which might be changed soon…)

I would say, Xen was ok – but the OVM Manager was horrible… hopefully that will change now with the new “OLVM”…

Hope we will see Solaris 11 x86 support as a guest soon!

locale dilemma

Found for me a new behavior when installing 11.4… some commands are complaining about my locale setting…

root@t7zone01:~# pkg history
Failed to set locale: unsupported locale setting.  Falling back to C.
pkg: Unable to set locale 'de_AT.UTF-8'; locale package may be broken or
not installed.  Reverting to C locale.

pkg: 'ascii' codec can't encode character u'\xdf' in position 8: ordinal not in range(128)
root@t7zone01:~#
root@t7zone01:~# locale
LANG=de_AT.UTF-8
LC_CTYPE="C"
LC_NUMERIC="C"
LC_TIME="C"
LC_COLLATE="C"
LC_MONETARY="C"
LC_MESSAGES="C"
LC_ALL=
root@t7zone01:~# 
root@dbc01:~# ls /usr/share/locale/
C                 de.UTF-8          de_DE.ISO8859-15  en@boldquot       en_US             it                pt_BR
de                de_DE             de_DE.UTF-8       en@quot           es                ja                zh_CN
de.ISO8859-15     de_DE.ISO8859-1   en                en@shaw           fr                ko                zh_TW
root@dbc01:~# 

Where does that de_AT comes from? During the installation I said I want default “C” nothing else… and its not installed
I found that de_AT on my client! I am running my laptop/mac with de_AT.UTF-8… sitting in Vienna/Austria 😉
Well, Solaris 11.4 comes with OpenSSH, not SunSSH anymore

pressy@PRESSY-MBP:~ # nmap solaris11.3 -p 22 -A
Starting Nmap 7.70 ( https://nmap.org ) at 2019-07-16 17:24 CEST
Nmap scan report for 10.52.72.60
Host is up (0.0035s latency).

PORT   STATE SERVICE VERSION
22/tcp open  ssh     SunSSH 2.4 (protocol 2.0)
| ssh-hostkey:
|   1024 85:d6:64:af:f3:14:1e:4b:86:2a:da:a2:6b:ba:6e:c9 (DSA)
|_  2048 50:bd:1d:d5:98:72:3f:ae:5f:ab:d6:10:a0:aa:8d:82 (RSA)

Service detection performed. Please report any incorrect results at https://nmap.org/submit/ .
Nmap done: 1 IP address (1 host up) scanned in 1.62 seconds
pressy@PRESSY-MBP:~ #
pressy@PRESSY-MBP:~ # nmap solaris11.4 -p 22 -A
Starting Nmap 7.70 ( https://nmap.org ) at 2019-07-16 17:25 CEST
Nmap scan report for 10.52.72.201
Host is up (0.0042s latency).

PORT   STATE SERVICE VERSION
22/tcp open  ssh     OpenSSH 7.7 (protocol 2.0)
| ssh-hostkey:
|   2048 27:ae:7e:e7:5f:70:d0:eb:b9:71:61:f3:eb:c5:bf:7d (RSA)
|_  256 0e:4d:9a:5d:83:00:5d:26:03:e2:1f:8e:3a:2e:ed:f9 (ED25519)

Service detection performed. Please report any incorrect results at https://nmap.org/submit/ .
Nmap done: 1 IP address (1 host up) scanned in 0.62 seconds
pressy@PRESSY-MBP:~ #
pressy@PRESSY-MBP:~ # locale
LANG="de_AT.UTF-8"
LC_COLLATE="de_AT.UTF-8"
LC_CTYPE="de_AT.UTF-8"
LC_MESSAGES="de_AT.UTF-8"
LC_MONETARY="de_AT.UTF-8"
LC_NUMERIC="de_AT.UTF-8"
LC_TIME="de_AT.UTF-8"
LC_ALL=
pressy@PRESSY-MBP:~ #

Found a little (for me “new”) settings in my ssh_config that was obviously ignored by SunSSH.

pressy@PRESSY-MBP:~ # grep SendEnv /etc/ssh/ssh_config
	SendEnv LANG LC_*
pressy@PRESSY-MBP:~ # 

OK -> many ways to fix it… but yes, that brought me to the point, how to install my “de_AT” on Solaris 11.4…

root@t7zone01:~# svccfg -s svc:/system/environment:init listprop environment/LANG
environment/LANG astring     C
root@t7zone01:~# nlsadm get-timezone
timezone=Europe/Vienna
root@dbc01:~# pkg search -r de_AT.UTF-8
Failed to set locale: unsupported locale setting.  Falling back to C.
pkg: Unable to set locale 'de_AT.UTF-8'; locale package may be broken or
not installed.  Reverting to C locale.

pkg: 'ascii' codec can't encode character u'\xdf' in position 8: ordinal not in range(128)
root@dbc01:~# export LC_ALL=C
root@dbc01:~# pkg search -r de_AT.UTF-8
INDEX      ACTION VALUE                                  PACKAGE
basename   link   usr/share/sysconfig/help/de_AT.UTF-8   pkg:/system/install/configuration@11.4-11.4.10.0.1.3.0
basename   link   usr/share/locale/de_AT.UTF-8           pkg:/system/locale@11.4-11.4.9.0.1.4.0
basename   file   usr/lib/locale/de_AT.UTF-8/de_AT.UTF-8 pkg:/system/locale@11.4-11.4.9.0.1.4.0
root@dbc01:~# nlsadm install-locale de_AT.UTF-8
Reading package information from IPS publisher's repository (this may take awhile) ...
Locale(s) to install:
 de_AT.ISO8859-15
 de_AT.ISO8859-1
 de_AT.UTF-8

Facet(s) to change:
 facet.locale.de_AT

Additional locales need to be also installed due to the structure of packaging.
Add the locales to the command argument or run with '-f/--force' option to install additional locales
root@dbc01:~#
root@dbc01:~# nlsadm install-locale -f de_AT.UTF-8 de_AT.ISO8859-1 de_AT.ISO8859-15
Reading package information from IPS publisher's repository (this may take awhile) ...
Locale(s) to install:
 de_AT.ISO8859-15
 de_AT.ISO8859-1
 de_AT.UTF-8

Facet(s) to change:
 facet.locale.de_AT

Updating installed packages and facets (this may take awhile) ...
Done
root@dbc01:~#

YES -> that’s it…
BUT….. oh my god… english/german mix on my cli:

root@dbc01:~# pkg history -n 2
START                    OPERATION                CLIENT             OUTCOME
2019-07-15T15:06:48      install                  pkg                Erfolgreich
2019-07-16T09:55:12      change-facet             pkg                Erfolgreich

OK -> so I removed my ssh_config setting on my mac 😉

Have FUN!

RAMBleed on Solaris/SPARC?

Using enterprise class hardware shows once again that you get what you paid for.

A new issue came up in the last days describing attacks against Dynamic Random Access Memory (DRAM) modules that are already susceptible to Rowhammer-style attacks. At the end of the day these security problems are not microprocessor-specific, they leverage known issues in DRAM memory. These attacks only impact DDR4 and DDR3 memory modules, and older generations DDR2 and DDR1 memory modules are not vulnerable to these attacks.
Oracle published an advisory that reported that older and current servers using its SPARC and x86 CPUs aren’t expected to be susceptible to RAMBleed:

RAMBleed

All current and many older families of Oracle x86 (X5, X6, X7, X8, E1) and Oracle SPARC servers (S7, T7, T8, M7, M8) employing DDR4 DIMMs are not expected to be impacted by RAMBleed. This is because Oracle only employs DDR4 DIMMs that have implemented the Target Row Refresh (TRR) defense mechanism against RowHammer. Oracle’s memory suppliers have stated that these implementations have been designed to be effective against RowHammer.

Just to remember once again… 😉

Meltdown@SPARC – NO
Spectre@SPARC – OKnot all variants (HW_BTI & OS fixed)
Foreshadow@SPARC – NO
ZombieLoad@SPARC – NO
RAMBleed@SPARC – OKvery difficult based on TRR DIMMs

Working in an enterprise data center and securing your business critical services might be easier than you thought… work with Solaris on SPARC!

No RISC no fun