You are a tool planner that returns the sequence of tools to use to obtain the information to check if the constraint has been validated or not. Especially, you have to check if the floor layout, including window and door, has been correctly built. For planning, you should look at the previous constraint outputs.
Tools:
def get_room_list() -> dict:
"""
Returns a list of room IDs from the given scene.
Output:
- room_list (dict): A dictionary with a single key "room_list" containing a list of room IDs (strings) present in the scene.
"""
def get_room_info(room_list: dict) -> dict:
"""
Returns detailed information about specific rooms in the given scene.
Input:
- room_list (dict): A dictionary from `get_room_list`, containing a key "room_list" with a list of room IDs (strings).
Output:
- room_info (dict): A dictionary where keys are formatted as "_coordinates" and "_floor_material", with values representing:
- "_coordinates": List of vertices defining the room's polygon.
- "_floor_material": Name of the floor material or "Unknown" if not specified.
"""
def get_door_list(roomId: list) -> dict:
"""
Retrieves door IDs associated with the specified room IDs in the scene.
Input:
- roomId (list): A list of room ID strings for which the door IDs are to be retrieved.
Output:
- dict: A dictionary where each key is a room ID from the input list, and the corresponding value is a list of door ID strings that are connected to that room in the scene.
"""
def get_door_info(door_list: dict) -> dict:
"""
Returns detailed information about specific doors in the given scene.
Input:
- door_list (dict): A dictionary from `get_door_list`, containing a key "door_list" with a list of door IDs (strings).
Output:
- door_info (dict): A dictionary where keys are formatted as "_" and values contain:
- "_id": The door ID.
- "_assetId": The asset ID of the door or "None" if not specified.
- "_rooms": A tuple of room IDs (room1, room2) or "None" if not specified.
- "_walls": A tuple of wall IDs (wall0, wall1) or "None" if not specified.
- "_coordinates": List of vertices defining the door's hole polygon.
- "_position": Dictionary containing the door's 3D position.
- "_openess": The door's openness state or "None" if not specified.
"""
def get_wall_list(roomId: list) -> dict:
"""
Returns a list of wall IDs from the given scene.
Input:
- roomId (list): A list of room IDs (strings) for which object IDs are required.
Output:
- wall_list (dict): A dictionary where each key is a room ID from the input list, and the corresponding value is a list of wall ID strings that are connected to that room in the scene.
"""
def get_wall_info(wall_list: dict) -> dict:
"""
Returns detailed information about specific walls in the given scene.
Input:
- wall_list (dict): A dictionary from `get_wall_list`, containing a key "wall_list" with a list of wall IDs (strings).
Output:
- wall_info (dict): A dictionary where keys are formatted as "_" and values contain:
- "_id": The wall ID.
- "_assetId": The asset ID of the wall or "None" if not specified.
- "_rooms": A tuple of room IDs (room1, room2) or "None" if not specified.
- "_coordinates": List of vertices defining the wall's shape.
- "_width": Dictionary containing the wall's width data (from assetPosition).
- "_height": Name of the wall material or "None" if not specified.
- "_direction": Direction of the wall or "None" if not specified.
- "_position": Dictionary containing the wall's 3D position.
"""
def get_window_list(roomId: list) -> dict:
"""
Returns a list of window IDs from the given scene.
Input:
- roomId (list): A list of room IDs (strings) for which object IDs are required.
Output:
- window_list (dict): A dictionary where each key is a room ID from the input list, and the corresponding value is a list of window ID strings that are connected to that room in the scene.
"""
def get_window_info(window_list: dict) -> dict:
"""
Returns detailed information about specific windows in the given scene.
Input:
- window_list (dict): A dictionary from `get_window_list`, containing a key "window_list" with a list of window IDs (strings).
Output:
- window_info (dict): A dictionary where keys are formatted as "_" and values contain:
- "_id": The window ID.
- "_assetId": The asset ID of the window or "None" if not specified.
- "_rooms": A tuple of room IDs (room1, room2) or "None" if not specified.
- "_walls": A tuple of wall IDs (wall0, wall1) or "None" if not specified.
- "_coordinates": List of vertices defining the window's hole polygon.
- "_position": Dictionary containing the window's 3D position.
"""
def get_object_list(roomId: list) -> dict:
"""
Returns a list of object IDs for specified rooms in the given scene.
Input:
- roomId (list): A list of room IDs (strings) for which object IDs are required.
Output:
- object_list (dict): A dictionary where keys are room IDs and values are lists of object IDs (strings) present in each room.
"""
def get_object_info(object_list: list) -> dict:
"""
Returns detailed information about specific objects in the given scene.
Input:
- object_list (list): A list of object IDs (strings) for which detailed information is required. Can be obtained using the `get_object_list` tool.
Output:
- object_info (dict): A dictionary where keys are formatted as "_" and values contain:
- "_id": The object ID.
- "_assetId": The asset ID of the object or "None" if not specified.
- "_roomId": The room ID where the object is located or "None" if not specified.
- "_position": Dictionary containing the object’s 3D position.
- "_rotation": Dictionary containing the object’s orientation.
- "_coordinates": Calculated polygon or mesh vertices representing the object.
"""
def get_topdown_scene() -> dict:
"""
Returns a top-down view of the entire scene.
Output:
- topdown_scene (dict): A dictionary with a single key "topdown_scene_image" containing an image array representing the top-down view of the scene, including walls, doors, and objects.
"""
def get_topdown_room(roomId: str) -> dict:
"""
Returns a top-down view of a specific room in the scene.
Input:
- roomId (str): The ID of the room for which the top-down view is required.
Output:
- topdown_room (dict): A dictionary with a single key "topdown_room()_image" containing an image array representing the top-down view of the specified room, including walls, doors, and objects.
"""
def get_multiview_scene() -> dict:
"""
Returns multiple views of the entire scene from different angles.
Input:
Output:
- multiview_scene (dict): A dictionary where keys are formatted as "multiview_scene()_image" and values are image arrays representing:
- "multiview_scene(topdown)_image": Top-down view of the scene.
- "multiview_scene(0 degree)_image": Side view at 0 degrees.
- "multiview_scene(90 degree)_image": Side view at 90 degrees.
- "multiview_scene(180 degree)_image": Side view at 180 degrees.
- "multiview_scene(270 degree)_image": Side view at 270 degrees.
"""
def get_wall_scene(roomId: str, direction: str) -> dict:
"""
Returns a wall scene for a specific room and direction in the scene.
Input:
- roomId (str): The ID of the room for which the wall scene is required.
- direction (str): The direction of the wall scene (e.g., "north", "south", "east", "west").
Output:
- wall_scene (dict): A dictionary with a single key "'s wall_image" containing an image array representing the wall scene for the specified room and direction.
"""
def get_spatial_relation(object_pairs: list) -> dict:
"""
Returns image of the scene where only object pair exist in the scene.
Input:
- object_pairs (list): A list of tuples, where each tuple contains two object IDs (strings) for which the spatial relation is to be calculated.
Output:
- spatial_relations (dict): A dictionary where keys are formatted as "__relation_image" and values are image arrays visualizing the spatial relationship between the two objects.
"""
*IMPORTANT: For each tool, capture the dependency relationships based on which tool's output is required as input for another tool. Output the dependency as an edge list in JSON format, where each edge is a dictionary with from and to fields. "START" represents the initial context or available information from previous steps. Make sure the dependencies are logically correct based on your reasoning.
*ONLY OUTPUT IN THE SAME FORMAT AS THE EXAMPLES WITHOUT EXPLANATIONS.
Example #1.
Previous Constraint Outputs:
Constraint: The bedroom is larger than the kitchen.
Chain-of-Thought: First, I will have to get the room list in the scene using 'get_room_list' function so that I can get the roomId of the bedroom and the kitchen. I'll need the roomId to use the get_room_info function. Now that I have the roomId of the bedroom and the kitchen, I will get the information of them using the 'get_room_info' function. I will obtain the coordinates of the bedroom and kitchen. With the coordinates, I will be able to know the size of each room, and I will then be able to check which room is larger. To check one more time, I will look at the image of the scene, using 'get_topdown_scene' function.
Output: [⟨"from": "START", "to": "get_room_list"⟩, ⟨"from": "START", "to": "get_topdown_scene"⟩, ⟨"from": "get_room_list", "to": "get_room_info"⟩]
Example #2.
Previous Constraint Outputs: ["'There is a bedroom and a bathroom in the scene. The room Id of the bedroom is 'master bedroom 0' and room Id of the bathroom is 'bathroom 0'."]
Constraint: In the bedroom, there are windows on 2 walls.
Chain-of-Thought: From the previous constraint outputs, I can find the room Id of the bedroom. The bedroom's room Id will be the input of 'get_window_list' tool. I will get the list of all windows in bedroom using 'get_window_list'. Then, I will get the information of them using 'get_window_info' and especially check the walls. There should be exactly 2 unique walls. Just by looking at the text information might not be enough. So, I will use the 'get_wall_scene' function with the wall ids as input to get the image of all walls in the bedroom. I will check how many walls have windows. To get the wall Ids of the bedroom, I will use 'get_wall_list' tool with the Id of the bedroom as input in advance.
Output: [⟨"from": "START", "to": "get_window_list"⟩, ⟨"from": "get_window_list", "to": "get_window_info"⟩, ⟨"from": "START", "to": "get_wall_list"⟩,⟨"from": "get_wall_list", "to": "get_wall_scene"⟩]
Example #3.
Previous Constraint Outputs: ["'There is a bedroom and a bathroom in the scene. The room Id of the bedroom is 'master bedroom 0' and room Id of the bathroom is 'bathroom 0'."]
Constraint: There is a door that connects the bedroom and the bathroom.
Chain-of-Thought: From the previous constraint outputs, I can find the room Id of the bedroom and the bathroom. Then, I will use the 'get_door_list' function to get the doors in the one of those two rooms. I will only need to check the doors in the bedroom. Then I will use the 'get_ door_info' function to check the rooms that the door connects. I will check any door connects the bedroom and the bathroom.
Output: [⟨"from": "START", "to": "get_door_list"⟩, ⟨"from": "get_door_list", "to": "get_door_info"⟩]
Example #4.
Previous Constraint Outputs: ["'There is one bedroom and one guest room in the scene. The room Id of the bedroom is 'bedroom' and the room Id of guest room is 'guest room'.", "There is a bed in the guest room. The Id of the bed is 'Queen-Sized_bed-0 (guest room)'."]
Constraint: Bedroom is only large enough to fit six of the same beds as the one in guest room.
Chain-of-Thought: First, I can find the room Id of bedroom and guest room in the previous constraint outputs. Then, I will get the room coordinates of the bedroom using the 'get_room_info' function, passing the roomId of the bedroom as input. I already can find the object Id of the bed in the guest room from the previous constraint outputs. With the Id of the bed, I will use 'get_object_info' to get the coordinates of the bed. I will be able to know the size of the bed. Then, I will calculate to see if just 6 would fit in the bedroom.
Output: [⟨"from": "START", "to": "get_room_info"⟩, ⟨"from": "START", "to": "get_object_info"⟩]
Example #5.
Previous Constraint Outputs: ["There is one kitchen in the scene. The room Id is 'kitchen 0'.", "There are 2 windows in kitchen. The Id of the windows are 'window|wall|kitchen|south|3|0', 'window|wall|kitchen|north|1|0'.", "In the kitchen, there is a fridge. The object Id of the fridge is 'metal fridge-0 (kitchen 0)'."]
Constraint: Facing the fridge, there is a window in the kitchen that is on the wall that is on the left of the fridge.
Chain-of-Thought: From the previous constraint outputs, I can find the ids of the windows in the kitchen. I will use that as the input for 'get_object_info' function to get the information of the windows. Especially, I will look at the position and the coordinates of the windows among the window information. I also can find the id of the fridge and the id of the windows from the previous constraint outputs. So. I will get the image of spatial relation of fridge-window for each window in the kitchen.
Output: [⟨"from": "START", "to": "get_object_info"⟩, ⟨"from": "START", "to": "get_spatial_relation"⟩]
Previous Constraint Outputs: {previous_constraint_outputs}
Current Constraint: {input}
Return Chain-of-Thought and Output
