Stuttering every 10-15s

Avatar
Rubyeyed
  • zaktualizowano
  • Completed - Resolved

Hi, i have these like frame skips/stutters every 10-15s and in mission 3 (HoG) they get more severe. What i mean is they are longer more like a second long which makes it very annoying and distracting. It happens regardless of the settings and what happens in the game. I can start a mission and move camera in circles around my core and it will still stutter every 10-15s. Although, in skirmish when the map is small and nothing much happening on it the stutters are so brief it is hard to notice them. On big maps with units etc. already on them (like HoG lvl 3) the stutters are longer and are a real pain.


Windows 7 x64

12gb RAM

i5 2500k @ 4.2

gtx 970

game version 1.4.0f4

output_log.txt

Game Version:
Steam Public
Platform:
Windows
Duplikaty 4
second to last mission unplayable due extreme lag

Every second or so the game just freezes/lags a bit. Its unpossible to finish the campaing. Im running I7-4770K @ 3.5GHz and GTX970 and 16GB ram.

Stuttering during play
I haven't played this in a while so with the release of the new DLC I decided to give it another bash, I've never completed it since low fps on later levels caused the game to become unplayable for me, less than 1fps.


I've built a completely new system since last time, wasn't expecting any issues but I've just completed the level to Defile Uthors Tomb and the last half of the level I kept stuttering every few seconds, I wouldn't think this should be an issue on my system at all but perhaps there is a fix?

My System:
i7-5820K
24GB DDR4 2666MHz
AMD R9 Fury-X

Windows 10

output_log.txt

error.log

Please fix the constant stuttering

I have a really powerful computer, i5, Gtx 970, 32 GB RAM and the game constantly stutters for a fraction of a second. This is really annoying and also often causes miss-clicks, or clicks not working, and is really immesion breaking.

I read that this is due to the engine you use and the garbage collection of that engine. I also read that you are wrapping up extra content for this game in the next patch. I bought this game twice. Once on steam and once from GOG because I really loved DK2 and I really love this game. You guys have done an amazing job and I wanted to support you. I also like my games DRM free. However please don't leave this game with such an annoying bug on an otherwise amazing game. This bug literally affects all campaign levels, skirmish, multiplayer etc.

Please fix this issue soon.

Freezes after 5-10min each 5-10 seconds getting worse overtime
  • Specify your platform
    • OS Windows
    • Version 10
    • 64bit
      • Ram 16GB
      • CPU FX8350
      • GPU GTX 760
  • Specify the build you are using (there will be a timestamp visible on the main menu)

        1.6.4

  • Include as much information as possible (what map you were on, screenshots, videos, etc)

        After about 5 to 10 minutes, a small freeze starts appearing every 5 or so seconds. This freeze gets progressively worse the longer you play. I have no issues in between those time frames, the game runs smooth.
There seems to be little to no effect if I change the graphical options from the highest to the lowest possible settings (while keeping it 1080p though). The map size does seem to matter on how fast the freezes start appearing, under 100x100 it can take up to 10minutes. On the larger ones, it'll start after ~5 minutes.


I feel like the main thread or game ticks are not getting enough resources. I see 2 cores basically at a 100% while the other 6 are mainly idling.


I've been struggling with the performance since for ever now. In the past I was able to play the smaller maps decently, now I can't comfortably play this game at all.



  • Search the Bug Tracker for any similar or identical open reports, if one exists comment there instead of opening a new ticket

       I did a search, didn't seem to find anything related to this.

Odpowiedzi 63
Avatar
0
Kanakotka

Hi. I mostly program in C#. While there are no tools like heap allocation and manual garbage removal (apart from forcing GC to do its thing) it's entirely down to the programmer to manage their garbage regardless of what they program in. To someone unfamiliar with garbage and its allocation, the hoops you have to jump through may seem very strange and even counter-intuitive in places, but the goal is to ensure minimal impact on GC Alloc in the profiler.


Essentially, there should never be per-frame allocation of garbage outside of obvious cases (streaming content, loading, certain usages of coroutines).


2)

All projects end up using multiple frameworks. Even though Unity tries to fill the gap, it is not "one program to rule them all" style of affair. Although what it has largely achieved is removal of dependency from using multiple languages to work on the project.


3)

Foreach is actually "improperly" handled by the .NET framework Unity uses. To be specific, it is improperly handled from the viewpoint of being used in a game engine; it is excellently handled from the viewpoint of a generic programming language. While patch 5.5(IIRC) made a workaround to prevent boxing usage in foreach clause, the end effect is a less efficient foreach. Keep in mind game programming is extremely abnormal when compared to normal, sequential programming.


You argue that it is improper for Unity's GC to throw away memory that's not being used, something that it's designed to do, such as a temporary variable access of parent or child of an object, yet any programmer should know to cache commonly used variables such as that. Imagine that you made a flexible regex algorithm and instead of simply writing "include System.Text.RegularExpression;" you argue that the compiler should instead arbitrarily understand to, without direction, cache the variable when you call for it each time with "new System.Text.RegularExpression.Regex".


It may seem counter-intuitive at first, but keep in mind Unity allows usage of these "improper methods" to make concepting faster. It's easy to forget to cache your variables in memory if they're used often. A very simple change in code will provide a massive performance improvement. This is not "because Unity", it's because how object-oriented languages and nesting works. It'll produce massive amount of garbage in a standard .NET program as well, but since an office program for instance doesn't need to update its view tens of times per second, it's not as visible. :)


4)

The Garbage Collector is handling the memory leak. Any GC alloc in the profiler is leaked memory. You have a severe memory leak if the GC is forced to run its routine more often than say, once every 5-10 minutes. In the later missions it's noticably being run every 20 seconds or less. The worst case so far was the mission before Mira's mission. (avoiding spoilers)


5)

Yes. While i'm not at the liberty to say which game, i can blurt out some ballpark numbers about 300,000+ lines of code and 20-25 developers not counting the QA.

The profiler doesn't need to run perpetually and you can create a test environment for its specific needs. From just a "fly in the ceiling" perspective, the major garbage producers seem to be related to creature behavior in some manner as the problem seems to compound with larger amount of creatures. Though a garbage issue should show itself in a small scale scenario as well in the profiler's GC Alloc listing. Ideally this should be <1kb / frame with 100 entities.


I'm afraid you misunderstood me. There are some very easy measures you can do right now listed in the link i posted, but those very easy measures are just the very tip of the iceberg. On the long run, there is no magic bullet for this unfortunately. The absolute best you can do is to follow the guide and see what comes up.


I hope for the best, because i really, really want to recommend the game onward. Despite the apparent negative tone of these posts, i absolutely adore what you're doing with it. I just really wish it didn't stutter, because that just takes me out of it so much. I guess this is more of an issue from a game dev standpoint, because it just makes me want to get the wrench out and come and tweak some bolts.

Avatar
0
[Dev] Nanorock Programmer

Hi Kanakotka.

The subject have been discussed countless time, and as such I will be brief here.

1) The game is coded in C#, so there is no "before proper GC"

2) Big project end up using multiple framework, which are black boxes that might generate GC without us easily knowing (Unity being one)

3) You mention foreach for example. It's such a simple language feature yet actually improperly handled by mono and produces GC. It shows that "the bed on which we lie" is not very comfy right. And the list on those incorrect handling goes forever. A simple event subscription ? Mono considers it needs to regenerate the full list of subscriber, producing GC. A simple access to direct property of Unity, like transform ? A GC is produced for marshaling the ref from their native code.

4) You mention leak. People have played for more than 4h straight without OOM. I'm not saying it is not possible to trigger one, I'm sure some unfortunate scenario might, but we can park this one at least.

5) Have you ever tried to profile a game of this scale with Unity tools ? Probably not by the sound of it. The profiling just become unresponsive after a certain amount of nested calls. And due to the code architecture we have, there is quickly too many nesting for it. Not to mention the constant crashing it produces. Or the fact that Memory snap never worked for us.


So we tried cleanup routine. There is no "new" in update, there is no foreach used anymore. We're using pooling as much as we can. We had optimization sprints. I'm not saying we're perfect. There is no doubt room for improvement. But due to the nature and the scale of the project, those point you mentioned are very far from being as simple as you make it sounds like.

Avatar
0
Kanakotka

Hi, Unity developer here. This is an issue because the Garbage Collector keeps firing far too often. There is no engine that has a garbage collector that can handle the amount of memory garbage that the game is producing, and there is no garbage collector in the universe that can handle a persistent memory leak. Garbage collector is there for the occasional garbage generated by single-run routines, it is not designed, and cannot handle per-frame produced garbage. The stutters happen because the garbage collector reaches a critical mass threshold and it dumps its heap.


In old terms before proper GC handling this would be referred to as a severe memory leak. The engine's robust nature and the massive amounts of memory and powerful CPUs we have nowadays compensates the issue, but it is still a severe issue. Your programmers, not those of Unity have to find out what is producing the garbage, and make it produce less garbage. Unfortunately there is absolutely no magic bullet for this. General rule of thumb is to move as much as possible away from Update loop, though the easiest thing to do is to ensure there are no foreach statements that fire very often, simply replace them with for statements which do not produce garbage. Another quick fix is to ensure there are no "new" keywords within update loop(s).


It's a great game otherwise, but the very janky state of the game's programming, especially this late to the release is severely hampering enjoyability. It stutters on a new generation i7 with 32gb RAM and 1080GTX so much that i have to restart the game after every mission because your programmers have not done proper cleanup routines. Unity comes with a profiler to figure out these sorts of issues easily.


If i had a week with the code i could fix the memory leaks, but i find it unlikely you'd give me access to the source. You should've done something about this way earlier instead of trying to shift the blame on Unity when you're, in an analogue, using a bed frame as a ladder.


Here's some reading about the subject: https://unity3d.com/learn/tutorials/topics/performance-optimization/optimizing-garbage-collection-unity-games

Avatar
0
anonymous

We have released a patch today that gives a small performance bump, it likely won't solve the issue but may make it more bearable, please let us know if it helps

Avatar
0
anonymous

We have released a patch today that gives a small performance bump, it likely won't solve the issue but may make it more bearable

Avatar
0
drivetheory

i know it's out of your hands, i've been following this issue for a while now, thank for the reply though!

Avatar
0
anonymous

Hey drivetheory,


Unfortunately we are still waiting for Unity to upgrade their garbage collector. This is something that is completely out of our hands, we will keep everyone up to date with any information we receive from Unity themselves.


I would recommend in the meantime for you to save the game and reload.

Avatar
0
drivetheory

i hope migration to the newer unity engine release and the problems of garbage collection is of the utmost importance...


i love this game but the hitching that starts to occur every 3 seconds after you're 45 minutes into a dungeon makes the game unplayable.


to this day i still haven't completed the heart of gold campaign because of this problem...

Avatar
0
anonymous

Well then we're just going to have to wait until the next major patch

Avatar
0
Antti

Makes no change to game performance in eather situation sadly. :/