drashna/ryujinx
1
0
Fork 0
Code Activity Actions

Memory Read/Write Tracking using Region Handles (#1272)

Browse source 
* WIP Range Tracking

- Texture invalidation seems to have large problems
- Buffer/Pool invalidation may have problems
- Mirror memory tracking puts an additional `add` in compiled code, we likely just want to make HLE access slower if this is the final solution.
- Native project is in the messiest possible location.
- [HACK] JIT memory access always uses native "fast" path
- [HACK] Trying some things with texture invalidation and views.

It works :)

Still a few hacks, messy things, slow things

More work in progress stuff (also move to memory project)

Quite a bit faster now.
- Unmapping GPU VA and CPU VA will now correctly update write tracking regions, and invalidate textures for the former.
- The Virtual range list is now non-overlapping like the physical one.
- Fixed some bugs where regions could leak.
- Introduced a weird bug that I still need to track down (consistent invalid buffer in MK8 ribbon road)

Move some stuff.

I think we'll eventually just put the dll and so for this in a nuget package.

Fix rebase.

[WIP] MultiRegionHandle variable size ranges

- Avoid reprotecting regions that change often (needs some tweaking)
- There's still a bug in buffers, somehow.
- Might want different api for minimum granularity

Fix rebase issue

Commit everything needed for software only tracking.

Remove native components.

Remove more native stuff.

Cleanup

Use a separate window for the background context, update opentk. (fixes linux)

Some experimental changes

Should get things working up to scratch - still need to try some things with flush/modification and res scale.

Include address with the region action.

Initial work to make range tracking work

Still a ton of bugs

Fix some issues with the new stuff.

* Fix texture flush instability

There's still some weird behaviour, but it's much improved without this. (textures with cpu modified data were flushing over it)

* Find the destination texture for Buffer->Texture full copy

Greatly improves performance for nvdec videos (with range tracking)

* Further improve texture tracking

* Disable Memory Tracking for view parents

This is a temporary approach to better match behaviour on master (where invalidations would be soaked up by views, rather than trigger twice)

The assumption is that when views are created to a texture, they will cover all of its data anyways. Of course, this can easily be improved in future.

* Introduce some tracking tests.

WIP

* Complete base tests.

* Add more tests for multiregion, fix existing test.

* Cleanup Part 1

* Remove unnecessary code from memory tracking

* Fix some inconsistencies with 3D texture rule.

* Add dispose tests.

* Use a background thread for the background context.

Rather than setting and unsetting a context as current, doing the work on a dedicated thread with signals seems to be a bit faster.

Also nerf the multithreading test a bit.

* Copy to texture with matching alignment

This extends the copy to work for some videos with unusual size, such as tutorial videos in SMO. It will only occur if the destination texture already exists at XCount size.

* Track reads for buffer copies. Synchronize new buffers before copying overlaps.

* Remove old texture flushing mechanisms.

Range tracking all the way, baby.

* Wake the background thread when disposing.

Avoids a deadlock when games are closed.

* Address Feedback 1

* Separate TextureCopy instance for background thread

Also `BackgroundContextWorker.InBackground` for a more sensible idenfifier for if we're in a background thread.

* Add missing XML docs.

* Address Feedback

* Maybe I should start drinking coffee.

* Some more feedback.

* Remove flush warning, Refocus window after making background context
This commit is contained in:
riperiperi 2020-10-16 21:18:35 +01:00 • committed by GitHub
parent f02791b20c
commit b4d8d893a4
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
50 changed files with 2973 additions and 441 deletions
Download patch file Download diff file Expand all files Collapse all files

141
Ryujinx.Graphics.Gpu/Image/TextureManager.cs
View file

@ -4,8 +4,8 @@ using Ryujinx.Graphics.Gpu.Image;
using Ryujinx.Graphics.Gpu.Memory;
using Ryujinx.Graphics.Gpu.State;
using Ryujinx.Graphics.Texture;
using Ryujinx.Memory.Range;
using System;
using System.Collections.Generic;
namespace Ryujinx.Graphics.Gpu.Image
{
@ -51,9 +51,6 @@ namespace Ryujinx.Graphics.Gpu.Image
private readonly AutoDeleteCache _cache;
private readonly HashSet<Texture> _modified;
private readonly HashSet<Texture> _modifiedLinear;
/// <summary>
/// The scaling factor applied to all currently bound render targets.
/// </summary>
@ -82,9 +79,6 @@ namespace Ryujinx.Graphics.Gpu.Image
_overlapInfo = new OverlapInfo[OverlapsBufferInitialCapacity];
_cache = new AutoDeleteCache();
_modified = new HashSet<Texture>(new ReferenceEqualityComparer<Texture>());
_modifiedLinear = new HashSet<Texture>(new ReferenceEqualityComparer<Texture>());
}
/// <summary>
@ -735,8 +729,9 @@ namespace Ryujinx.Graphics.Gpu.Image
for (int index = 0; index < overlapsCount; index++)
{
Texture overlap = _textureOverlaps[index];
TextureViewCompatibility overlapCompatibility = overlap.IsViewCompatible(info, size, out int firstLayer, out int firstLevel);
if (overlap.IsViewCompatible(info, size, out int firstLayer, out int firstLevel) == TextureViewCompatibility.Full)
if (overlapCompatibility == TextureViewCompatibility.Full)
{
if (!isSamplerTexture)
{
@ -745,7 +740,7 @@ namespace Ryujinx.Graphics.Gpu.Image
texture = overlap.CreateView(info, sizeInfo, firstLayer, firstLevel);
if (IsTextureModified(overlap))
if (overlap.IsModified)
{
texture.SignalModified();
}
@ -759,6 +754,11 @@ namespace Ryujinx.Graphics.Gpu.Image
}
break;
}
else if (overlapCompatibility == TextureViewCompatibility.CopyOnly)
{
// TODO: Copy rules for targets created after the container texture. See below.
overlap.DisableMemoryTracking();
}
}
@ -849,7 +849,7 @@ namespace Ryujinx.Graphics.Gpu.Image
// Inherit modification from overlapping texture, do that before replacing
// the view since the replacement operation removes it from the list.
if (IsTextureModified(overlap))
if (overlap.IsModified)
{
texture.SignalModified();
}
@ -859,8 +859,13 @@ namespace Ryujinx.Graphics.Gpu.Image
// If the texture is a 3D texture, we need to additionally copy any slice
// of the 3D texture to the newly created 3D texture.
if (info.Target == Target.Texture3D)
if (info.Target == Target.Texture3D && viewCompatible > 0)
{
// TODO: This copy can currently only happen when the 3D texture is created.
// If a game clears and redraws the slices, we won't be able to copy the new data to the 3D texture.
// Disable tracking to try keep at least the original data in there for as long as possible.
texture.DisableMemoryTracking();
for (int index = 0; index < viewCompatible; index++)
{
Texture overlap = _textureOverlaps[index];
@ -872,7 +877,7 @@ namespace Ryujinx.Graphics.Gpu.Image
overlap.HostTexture.CopyTo(texture.HostTexture, oInfo.FirstLayer, oInfo.FirstLevel);
if (IsTextureModified(overlap))
if (overlap.IsModified)
{
texture.SignalModified();
}
@ -886,8 +891,6 @@ namespace Ryujinx.Graphics.Gpu.Image
if (!isSamplerTexture)
{
_cache.Add(texture);
texture.Modified += CacheTextureModified;
texture.Disposed += CacheTextureDisposed;
}
lock (_textures)
@ -901,42 +904,65 @@ namespace Ryujinx.Graphics.Gpu.Image
}
/// <summary>
/// Checks if a texture was modified by the host GPU.
/// Tries to find an existing texture matching the given buffer copy destination. If none is found, returns null.
/// </summary>
/// <param name="texture">Texture to be checked</param>
/// <returns>True if the texture was modified by the host GPU, false otherwise</returns>
public bool IsTextureModified(Texture texture)
/// <param name="tex">The texture information</param>
/// <param name="cbp">The copy buffer parameters</param>
/// <param name="swizzle">The copy buffer swizzle</param>
/// <param name="linear">True if the texture has a linear layout, false otherwise</param>
/// <returns>A matching texture, or null if there is no match</returns>
public Texture FindTexture(CopyBufferTexture tex, CopyBufferParams cbp, CopyBufferSwizzle swizzle, bool linear)
{
return _modified.Contains(texture);
}
ulong address = _context.MemoryManager.Translate(cbp.DstAddress.Pack());
/// <summary>
/// Signaled when a cache texture is modified, and adds it to a set to be enumerated when flushing textures.
/// </summary>
/// <param name="texture">The texture that was modified.</param>
private void CacheTextureModified(Texture texture)
{
texture.IsModified = true;
_modified.Add(texture);
if (texture.Info.IsLinear)
if (address == MemoryManager.BadAddress)
{
_modifiedLinear.Add(texture);
return null;
}
}
/// <summary>
/// Signaled when a cache texture is disposed, so it can be removed from the set of modified textures if present.
/// </summary>
/// <param name="texture">The texture that was diosposed.</param>
private void CacheTextureDisposed(Texture texture)
{
_modified.Remove(texture);
int bpp = swizzle.UnpackDstComponentsCount() * swizzle.UnpackComponentSize();
if (texture.Info.IsLinear)
int addressMatches = _textures.FindOverlaps(address, ref _textureOverlaps);
for (int i = 0; i < addressMatches; i++)
{
_modifiedLinear.Remove(texture);
Texture texture = _textureOverlaps[i];
FormatInfo format = texture.Info.FormatInfo;
if (texture.Info.DepthOrLayers > 1)
{
continue;
}
bool match;
if (linear)
{
// Size is not available for linear textures. Use the stride and end of the copy region instead.
match = texture.Info.IsLinear && texture.Info.Stride == cbp.DstStride && tex.RegionY + cbp.YCount <= texture.Info.Height;
}
else
{
// Bpp may be a mismatch between the target texture and the param.
// Due to the way linear strided and block layouts work, widths can be multiplied by Bpp for comparison.
// Note: tex.Width is the aligned texture size. Prefer param.XCount, as the destination should be a texture with that exact size.
bool sizeMatch = cbp.XCount * bpp == texture.Info.Width * format.BytesPerPixel && tex.Height == texture.Info.Height;
bool formatMatch = !texture.Info.IsLinear &&
texture.Info.GobBlocksInY == tex.MemoryLayout.UnpackGobBlocksInY() &&
texture.Info.GobBlocksInZ == tex.MemoryLayout.UnpackGobBlocksInZ();
match = sizeMatch && formatMatch;
}
if (match)
{
return texture;
}
}
return null;
}
/// <summary>
@ -1084,38 +1110,6 @@ namespace Ryujinx.Graphics.Gpu.Image
info.SwizzleA);
}
/// <summary>
/// Flushes all the textures in the cache that have been modified since the last call.
/// </summary>
public void Flush()
{
foreach (Texture texture in _modifiedLinear)
{
if (texture.IsModified)
{
texture.Flush();
}
}
_modifiedLinear.Clear();
}
/// <summary>
/// Flushes the textures in the cache inside a given range that have been modified since the last call.
/// </summary>
/// <param name="address">The range start address</param>
/// <param name="size">The range size</param>
public void Flush(ulong address, ulong size)
{
foreach (Texture texture in _modified)
{
if (texture.OverlapsWith(address, size) && texture.IsModified)
{
texture.Flush();
}
}
}
/// <summary>
/// Removes a texture from the cache.
/// </summary>
@ -1142,7 +1136,6 @@ namespace Ryujinx.Graphics.Gpu.Image
{
foreach (Texture texture in _textures)
{
_modified.Remove(texture);
texture.Dispose();
}
}