Previous investigations have suggested that the simplest spin-orbital model on the simplest frustrated lattice can host a nematic quantum spin-orbital liquid state. Namely, the orbital degeneracy of the SU(4) Kugel-Khomskii model tends to enhance quantum fluctuations and stabilize a quantum spin-orbital liquid exhibiting stripy features on the triangular lattice, as revealed by the state-of-the-art method of the density matrix renormalization group boosted by Gutzwiller projected wave functions. In this work, using the variational quantum Monte Carlo method, we study several spin-orbital liquid states, including a uniform $\ensuremath{\pi}$-flux state, three stripy states, and a plaquette state, on the $L\ifmmode\times\else\texttimes\fi{}L$ torus up to
L=24. It turns out that one of these stripy states, called the ``stripe-II'' state, is energetically favored. This ground state breaks the
C6 symmetry of the lattice, resulting in a reduced
C2 symmetry and doubled unit cells while preserving the SU(4) spin-orbital rotation symmetry. Such a nematic quantum spin-orbital liquid state can be characterized by a parton Fermi surface (FS) consisting of open orbits in the Brillouin zone, in contrast to the circular FS of the uniform $\ensuremath{\pi}$-flux state.
By combining the density matrix renormalization group (DMRG) method with Gutzwiller projected wave functions, we provide clear evidence that the ground state of the SU(4) Kugel-Khomskii spin-orbital model on the triangular lattice can be well described by a ``single Gutzwiller projected wave function with an emergent parton Fermi surface, despite it exhibits strong finite size effect and even-odd discrepancy in quasi-one-dimensional cylinders. This ground state preserves SU(4) symmetry, but spontaneously breaks translational symmetry by doubling the unit cell along one of the lattice vector directions. The finite size effect and even-odd discrepancy can be resolved by the fact that the parton Fermi surface consists of open orbits in the reciprocal space. Thereby, a nematic spin-orbital liquid state is expected in the two-dimensional limit. Furthermore, our DMRG results indicate that the fluctuating stripes are critical and the central charge of each stripe is
c=3, in agreement with the SU(4)
1 Wess-Zumino-Witten conformal field theory. All these results are consistent with the Lieb-Schultz-Mattis-Oshikawa-Hastings theorem.
Motivated by the absence of a cooperative Jahn-Teller effect in
LiNiO2 and
BaVS3, two layered oxides with triangular planes, we study the SU(4) symmetric spin-orbital model on the triangular lattice. Upon reducing the next-nearest-neighbor coupling, we show that the system undergoes a quantum phase transition to a liquid phase. A variational approach to this liquid phase shows that simple types of long-range correlations are suppressed, suggesting that it is stable against lattice distortions.