<p>Manifold aware <code>pytorch.optim</code>.</p>

<p>Unofficial implementation for <a href="https://openreview.net/forum?id=r1eiqi09K7" rel="noopener noreferrer" target="_blank">&quot;Riemannian Adaptive Optimization&nbsp;</a><a href="https://openreview.net/forum?id=r1eiqi09K7" rel="noopener noreferrer" target="_blank">Methods&quot;</a>&nbsp;at ICLR2019 and more.</p>

<p>Guthub page: <a href="https://github.com/geoopt/geoopt" rel="noopener noreferrer" target="_blank">https://github.com/geoopt/geoopt</a></p>

<h2>Installation</h2>

<ol>
<li>GitHub (preferred so far) due to active development
<pre>
<code>pip install git+https://github.com/geoopt/geoopt.git</code></pre>
</li>
<li>pypi (this might be significantly behind master branch)
<pre>
<code>pip install geopt</code></pre>

<ol>
</ol>
</li>
</ol>

<p>The preferred way to install geoopt will change once stable project stage is achieved.&nbsp;Now, pypi is behind master as we actively develop and implement new features.</p>

<h2>PyTorch Support</h2>

<p>Geoopt officially supports <strong>2 latest stable versions</strong> of pytorch upstream or the latest major release.</p>

<h2>What is done so far?</h2>

<p>Work is in progress but you can already use this. Note that API might&nbsp;change in future releases.</p>

<h4>Tensors</h4>

<ul>
<li>
<p><code>geoopt.ManifoldTensor</code> -- just as torch.Tensor with additional <code>manifold</code> keyword argument.</p>
</li>
<li>
<p><code>geoopt.ManifoldParameter</code>&nbsp;-- same as above, recognized in torch.nn.Module.parameters as correctly subclassed.</p>
</li>
</ul>

<p>All above containers have special methods to work with them as with&nbsp;points on a certain manifold</p>

<ul>
<li><code>.proj_()</code>&nbsp;--&nbsp;inplace projection on the manifold.</li>
<li><code>.proju(u)</code>&nbsp;--&nbsp;project vector <code>u</code> on the tangent space. You need&nbsp;to project all vectors for all methods below.</li>
<li><code>.egrad2rgrad(u)</code>&nbsp;--&nbsp;project gradient <code>u</code> on Riemannian manifold</li>
<li><code>.inner(u, v=None)</code>--&nbsp;inner product at this point for two&nbsp;<strong>tangent</strong> vectors at this point. The passed vectors are not<br />
&nbsp; projected, they are assumed to be already projected.</li>
<li><code>.retr(u)</code>--&nbsp;retraction map following vector <code>u</code></li>
<li><code>.expmap(u)</code>--&nbsp;exponential map following vector <code>u</code> (if expmap is not available in closed form, best approximation is used)</li>
<li><code>.transp(v,u)</code>--&nbsp;transport vector <code>v</code> &nbsp;with direction <code>u</code></li>
<li><code>.retr_transp(v,u)</code>--&nbsp;transport <code>self</code>, vector <code>v&nbsp;</code>(and possibly more vectors) with direction <code>u</code>&nbsp;(returns are plain tensors)</li>
</ul>

<h3>Manifolds</h3>

<ul>
<li><code>geoopt.Euclidean</code> -- unconstrained manifold in <code>R</code> with&nbsp;Euclidean metric</li>
<li><code>geoopt.Stiefel</code> --&nbsp;Stiefel manifold on matrices&nbsp;&nbsp;<code>A in R^{n x p} : A^t A=I, n &gt;= p</code></li>
<li><code>geoopt.Sphere</code> -- Sphere manifold <code>||x||=1</code></li>
<li><code>geoopt.BirkhoffPolytope</code> --&nbsp;manifold of Doubly Stochastic matrices</li>
<li><code>geoopt.Stereographic</code> --&nbsp;Constant curvature stereographic projection model</li>
<li><code>geoopt.SphereProjection</code> --&nbsp;Sphere stereographic projection model</li>
<li><code>geoopt.PoincareBall</code> --&nbsp;<a href="https://en.wikipedia.org/wiki/Poincar%C3%A9_disk_model" rel="noopener noreferrer" target="_blank">Poincare ball model</a></li>
<li><code>geoopt.Lorentz</code> --&nbsp;<a href="https://en.wikipedia.org/wiki/Hyperboloid_model" rel="noopener noreferrer" target="_blank">Hyperboloid model</a></li>
<li><code>geoopt.ProductManifold</code> --&nbsp;Product manifold constructor</li>
<li><code>geoopt.Scaled</code> --&nbsp;Scaled version of the manifold. Similar to <a href="https://openreview.net/forum?id=HJxeWnCcF7" rel="noopener noreferrer" target="_blank">Learning Mixed-Curvature Representations in Product Spaces</a> if combined with <code>ProductManifold</code></li>
<li><code>geoopt.SymmetricPositiveDefinite</code> --&nbsp;SPD matrix manifold</li>
<li><code>geoopt.UpperHalf</code> --&nbsp;Siegel Upper half manifold. Supports Riemannian and Finsler metrics, as in <a href="https://arxiv.org/abs/2106.04941" rel="noopener noreferrer" target="_blank">Symmetric Spaces for Graph Embeddings: A Finsler-Riemannian Approach</a></li>
<li><code>geoopt.BoundedDomain</code> --&nbsp;Siegel Bounded domain manifold. Supports Riemannian and Finsler metrics.</li>
</ul>

<p>All manifolds implement methods necessary to manipulate tensors on manifolds and&nbsp;tangent vectors to be used in general purpose. See more in `documentation`.</p>

<h2>Optimizers</h2>

<ul>
<li><code>geoopt.optim.RiemannianSGD</code> --&nbsp;a subclass of <code>torch.optim.SGD</code>&nbsp;with the same API and a Sparse version</li>
<li><code>geoopt.optim.RiemannianAdam&nbsp;</code>--&nbsp;a subclass of <code>torch.optim.Adam</code> and a Sparse version</li>
</ul>

<h2>Samplers</h2>

<ul>
<li><code>geoopt.samplers.RSGLD</code> -- Riemannian Stochastic Gradient Langevin&nbsp;Dynamics</li>
<li><code>geoopt.samplers.RHMC</code> --&nbsp;Riemannian Hamiltonian Monte-Carlo</li>
<li><code>geoopt.samplers.SGRHMC</code> --&nbsp;Stochastic Gradient Riemannian&nbsp;Hamiltonian Monte-Carlo</li>
</ul>

<h2><br />
Citing Geoopt</h2>

<p>If you find this project useful in your research, please kindly add this bibtex entry in references and cite.</p>

<pre>
<code>@misc{geoopt2020kochurov,</code>
<code>    title={Geoopt: Riemannian Optimization in PyTorch},</code>
<code>    author={Max Kochurov and Rasul Karimov and Serge Kozlukov},</code>
<code>    year={2020},</code>
<code>    eprint={2005.02819},</code>
<code>    archivePrefix={arXiv},</code>
<code>    primaryClass={cs.CG}</code>
<code>}</code></pre>

Maxim Kochurov

PyTorch framework for Riemannian Deep Learning

GeooptFounded 2 years ago

Geoopt
Founded 2 years ago