Connecting Dungeon Rooms Seamlessly Through Doors

How I simplified dungeon generation by focusing on organic door connections, leaving complicated grids behind.

Dungeon generation is a technique used in many games to create levels dynamically rather than designing them manually. A common method involves placing rooms randomly on a rigid grid, then connecting them with paths. This works, but often limits the layout’s creativity and flexibility. To make everything connect logically, you might need complex algorithms like Dijkstra's. But what if you wanted more freedom, diversity, and organic layouts?

I faced this exact problem and decided to experiment with something different. Instead of starting with grids, I thought, "Why not connect rooms organically through their doors?" Using generic room templates (prefabs) with predefined doors, each new room connects naturally to the previous one by selecting matching doors. It turned out to be simpler and more flexible than expected.

I wanted the whole process to be generic but simple. Tiles in the room template were marked by function: potential doors, solid walls, windows, traps, and basic floor tiles. This gave me a flexible way to quickly generate unique room structures without micromanaging every tile while also allowing for rooms to take up a shape, but be different.

The output includes some other systems, like the objects in the scene I will go over in another post, but this is what that ugly colored room comes out to be when generated during runtime.

In this system, doors act as our extension points. Unlike traditional dungeon generation, which places rooms first and connects them later with tiles or corridors, we flip that logic. We place a room, choose a door, then select and align another room based on that door. Throughout the process, we maintain a collection of "orphan" doors and unused connectors, which allows the system to branch naturally in different directions; This results in randomized, organic layouts without relying on complex pathfinding algorithms.

Despite the simplicity of the idea, I did hit some interesting challenges along the way:

How to consistently pick the "best" door to use next?
How to determine the ideal rotation between connecting rooms?
How to accurately calculate positional offsets to snap rooms together perfectly?

Selecting the optimal door was trickier than anticipated. Each room had multiple doors (with placement being random and unique, each positioned differently, and rooms could rotate in increments of 90°, adding complexity. My solution was to create a simple but effective "door scoring" method based on distance.

Test rotations: Temporarily rotate the new room through all possibilities (0°, 90°, 180°, 270°).
Measure distances: For each rotation, calculate distances from doors on the new room to the target door on the existing room.
Select the best: Choose the door and rotation combination with the shortest distance.

While effective, the method isn't perfect yet. It doesn't account for more complex room shapes, like "U-shaped" rooms connecting awkwardly to "T-shaped" rooms. However, it's a strong foundation and more than adequate for most needs.

After choosing doors, I needed to ensure rooms aligned logically. This meant accurately rotating rooms so doors faced each other naturally.

The rotation logic is straightforward:

If doors face opposite directions, rotate Room B by 180°.
If doors are perpendicular, rotate Room B by ±90°, choosing the shortest rotation.
If doors already face each other, no rotation is necessary.

Door A Orientation	Door B Orientation	Rotation Applied
Opposite Directions	(e.g., North & South)	180°
Perpendicular (Clockwise)	(e.g., North & East)	-90°
Perpendicular (Reverse)	(e.g., East & North)	90°
Already Aligned	(e.g., North & North)	0° (No rotation)

After aligning rotations, the final step was precisely positioning rooms so they'd "snap" neatly into place without gaps or overlaps. This part ate up more time than the rest of the system combined. I ran into countless edge cases, figuring out how to use door offsets, room bounds, and directional alignment was a math puzzle I had not dealt with before. Every fix introduced a new bug. But once I finally got it stable, everything else started falling into place, and just instantly worked.

The offset calculation involves:

Room Bounds: Using bounding boxes to understand the size and extents of each room.
Door Positions: Calculating exact differences between the connected doors.
Directional Logic: Using the door's orientation to determine exactly where the room should go.
- Up: Position next to Room A's right side.
- Right: Place in front of Room A.
- Down: Position next to Room A's left side.
- Left: Place behind Room A.

Here is an example code-block on how this simple function is the heart of the maze layout. I don't come from a math background, and most of my experience to this point was in web/software. So this math took me quite some time.


public static Bounds BoundingBox(this Transform transform, bool noChildBoundingBox = false)
{
    Transform target = noChildBoundingBox ? transform : transform.Find("BoundingBox");

    if (target == null)
    {
        Debug.LogError($"{transform.name} does not have a BoundingBox child.");
        throw new System.ArgumentException("Transform must have a BoundingBox child.");
    }

    // This is a CAST using arrows but this CODE element breaks the page if I use arrows.
    if (target.TryGetComponent[Renderer](out Renderer renderer))
    {
        return renderer.bounds;
    }

    Debug.LogWarning($"{target.name} does not have a Renderer component.");
    throw new System.ArgumentException("BoundingBox does not have a Renderer. Use a 3D mesh with a Renderer component.");
}

private static Vector3 GetRoomPositionWithCalculatedOffset(GameObject roomA, GameObject roomB, Transform roomADoor, Transform roomBDoor)
{
    Bounds roomABounds = roomA.transform.BoundingBox();
    Bounds roomBBounds = roomB.transform.BoundingBox();

    // Get piece options to grab door direction.
    RoomFixtureMono pieceA = roomADoor.GetComponentRoomFixtureMono();

    Vector3 doorOffset = (roomADoor.transform.position - roomBDoor.transform.position);

    float baseX = roomABounds.center.x + doorOffset.x;
    float baseY = roomABounds.center.y + doorOffset.y;
    float baseZ = roomABounds.center.z + doorOffset.z;

    switch (pieceA.Direction)
    {
        // +X.
        case SpatialOrientation.Up:
            return new Vector3(roomABounds.max.x + roomBBounds.extents.x, baseY, baseZ);

        // -Z
        case SpatialOrientation.Right:
            return new Vector3(baseX, baseY, roomABounds.min.z - roomBBounds.extents.z);

        // -X
        case SpatialOrientation.Down:
            return new Vector3(roomABounds.min.x - roomBBounds.extents.x, baseY, baseZ);

        // +Z
        case SpatialOrientation.Left:
            return new Vector3(baseX, baseY, roomABounds.max.z + roomBBounds.extents.z);

        // Invalid piece type.
        default:
            throw new System.ArgumentOutOfRangeException(
                $"Provided an invalid direction for calculating a room offset. Provided: {pieceA.Direction}");
    }
}

Here is some output images of the maze generation with small systems.

This was the start of an almost year-long project for me. I started this project after playing the hit game, Lethal Company. I found the mazes basic and limited. At this point, I had no experience in game development, and this was my first "idea".

I built this system in Unity, and it was just one part of many systems I created to extend the overall design. Everything came from a point of learning, along with substantial optimizations. Some of the other systems I hope to write about in future posts include:

Rebuilding the system to follow a simple grid tile system (required for further optimizations)
A 2D (and then 3D) hallway system to give orphan doors life
A procedural stairway system to support multi-level navigation
Simple A* pathfinding to help with basic AI

This project is hosted on my GitHub, which you can see here. It’s still in active development as I continue building new features and improving clarity and performance.

Until then, thanks for making it to the end!

Austin Douglas

Connecting Dungeon Rooms Seamlessly Through Doors

The door score system

Determining the best rotation between room A and room B

Determining the room offset