You are an argument selector of the tools. You must look at the reasoning of the sequence of tools, and choose the argument appropriately. You also have must look at the previous tool outputs and previous constraint outputs, because it will help you choose exact inputs. The output should be a list, containing Id(str) of rooms, doors, windows, walls, or objects as elements.
Tools:
def get_room_info(room_list: dict) -> dict:
"""
Returns detailed information about specific rooms in the given scene.
Input:
- room_list (dict): A dictionary from `get_room_list`, containing a key "room_list" with a list of room IDs (strings).
Output:
- room_info (dict): A dictionary where keys are formatted as "_coordinates" and "_floor_material", with values representing:
- "_coordinates": List of vertices defining the room's polygon.
- "_floor_material": Name of the floor material or "Unknown" if not specified.
"""
def get_door_info(door_list: dict) -> dict:
"""
Returns detailed information about specific doors in the given scene.
Input:
- door_list (dict): A dictionary from `get_door_list`, containing a key "door_list" with a list of door IDs (strings).
Output:
- door_info (dict): A dictionary where keys are formatted as "_" and values contain:
- "_id": The door ID.
- "_assetId": The asset ID of the door or "None" if not specified.
- "_rooms": A tuple of room IDs (room1, room2) or "None" if not specified.
- "_walls": A tuple of wall IDs (wall0, wall1) or "None" if not specified.
- "_coordinates": List of vertices defining the door's hole polygon.
- "_position": Dictionary containing the door's 3D position.
- "_openess": The door's openness state or "None" if not specified.
"""
def get_window_info(window_list: dict) -> dict:
"""
Returns detailed information about specific windows in the given scene.
Input:
- window_list (dict): A dictionary from `get_window_list`, containing a key "window_list" with a list of window IDs (strings).
Output:
- window_info (dict): A dictionary where keys are formatted as "_" and values contain:
- "_id": The window ID.
- "_assetId": The asset ID of the window or "None" if not specified.
- "_rooms": A tuple of room IDs (room1, room2) or "None" if not specified.
- "_walls": A tuple of wall IDs (wall0, wall1) or "None" if not specified.
- "_coordinates": List of vertices defining the window's hole polygon.
- "_position": Dictionary containing the window's 3D position.
"""
def get_wall_info(wall_list: dict) -> dict:
"""
Returns detailed information about specific walls in the given scene.
Input:
- wall_list (dict): A dictionary from `get_wall_list`, containing a key "wall_list" with a list of wall IDs (strings).
Output:
- wall_info (dict): A dictionary where keys are formatted as "_" and values contain:
- "_id": The wall ID.
- "_assetId": The asset ID of the wall or "None" if not specified.
- "_rooms": A tuple of room IDs (room1, room2) or "None" if not specified.
- "_coordinates": List of vertices defining the wall's shape.
- "_width": Dictionary containing the wall's width data (from assetPosition).
- "_height": Name of the wall material or "None" if not specified.
- "_direction": Direction of the wall or "None" if not specified.
- "_position": Dictionary containing the wall's 3D position.
"""
def get_object_info(object_list: list) -> dict:
"""
Returns detailed information about specific objects in the given scene.
Input:
- object_list (list): A list of object IDs (strings) for which detailed information is required. Can be obtained using the `get_object_list` tool.
Output:
- object_info (dict): A dictionary where keys are formatted as "_" and values contain:
- "_id": The object ID.
- "_assetId": The asset ID of the object or "None" if not specified.
- "_roomId": The room ID where the object is located or "None" if not specified.
- "_position": Dictionary containing the object’s 3D position.
- "_rotation": Dictionary containing the object’s orientation.
- "_coordinates": Calculated polygon or mesh vertices representing the object.
"""
ONLY OUTPUT IN THE SAME FORMAT AS THE EXAMPLES WITHOUT EXPLANATIONS.
Example 1.
User Instruction: There are two rooms in the scene. I want both rooms' floor materials to be the same. Both rooms should also be the same size.
Previous Constraint Outputs: ["There are two rooms in the scene, and the room Id of the first room is 'room 0', and the other room's Id is 'room 1'."]
Current Constraint: Both rooms have same floor materials.
Tool Sequence: ['get_room_info']
Reasoning behind Tool Sequence: I got the room Ids of the both rooms from the previous constraint, which are 'room 0
and 'room 1'. Using those room IDs as input, I will use the 'get_room_info' tool to get the information of the rooms. I will especially look at the floor materials of each room.
Previous Tool Outputs:
Current Tool to Use: 'get_room_info'
Arguments: ['room 0', 'room 1']
Example 2.
User Instruction: There is one bedroom and one bathroom in the scene. In the bedroom, there is a door that leads to the exterior. That door is larger than 1m x 1m. In the bathroom, there is a mirror. In the bedroom, there is a desk. On the desk, there is a pencil.
Previous Constraint Outputs: ["The room Id of the bedroom is 'bedroom 0', and the room Id of the bathroom is 'bathroom 0'.", "The id of the door that leads to the exterior is 'door|0|exterior|bedroom'."]
Current Constraint: The door that leads to the exterior, which is in the bedroom, is brown.
Tool Sequence: ['get_door_info']
Reasoning behind Tool Sequence: From the previous constraint output, I already know that the id of the door that leads to the exterior, which is in the bedroom, is 'door|0|exterior|bedroom 0'. I will input that to the 'get_door_info' tool so that I can get the information of the door. I will especially look at the coordinates of the door to see if it exceeds 1m x 1m.
Previous Tool Outputs:
Current Tool to Use: 'get_door_info'
Arguments: ['door|0|exterior|bedroom 0']
Example 3.
User Instruction: There is a bedroom and a bathroom in the scene. In the bedroom, there are two windows. All two windows in the bedroom are on different walls. There are 3 couches in the bedroom. They are all orange. They are all shaped differently.
Previous Constraint Outputs: ["There is a bedroom and a bathroom in the scene, and their Ids are 'bedroom 0' and 'bathroom 0'.", "The Id of the windows in the bedroom are 'window|wall|bedroom 0|north|1|0' and 'window|wall|bedroom 0|east|3|0'."]
Current Constraint: All two windows in the bedroom are on different walls.
Tool Sequence: ['get_window_info', 'get_wall_scene']
Reasoning behind Tool Sequence: From the previous constraint output, I know the Ids of the windows in the bedroom. With the Ids as input, I will use the 'get_window_info' tool to get the information of those windows. I will be able to know on which wall they are on by looking at the information. Also, I will look at the scene walls, for all four walls. This is to make sure that I don't only rely on the text information that could be wrong.
Previous Tool Outputs:
Current Tool to Use: 'get_window_info'
Arguments: ['window|wall|bedroom 0|north|1|0', 'window|wall|bedroom 0|east|3|0']
Example 4.
User Instruction: I want to create one classroom in the scene. The two walls in the classroom are blue. The other two are red.
Previous Constraint Outputs: ["There is one classroom in the scene, and the Id is 'classroom 0'."]
Current Constraint: The two walls in the classroom are blue.
Tool Sequence: ['get_wall_list', 'get_wall_info', 'get_material_image', 'get_wall_scene']
Reasoning behind Tool Sequence: I already know the room Id of the classroom from the previous constraint output. With the room Id as input, I will use the 'get_wall_list' to get the list of walls(wall IDs) in the classroom. Then, with the wall ID list, I will use the 'get_wall_info' tool to find out the material of the walls. Then, I will use the 'get_material_image' to visually see if two walls are blue. I will also use the 'get_wall_scene' to visually see if two walls are actually blue in scene.
Previous Tool Outputs: ⟨' classroom 0_wall_list': ['wall|classroom 0|west|0', 'wall|classroom 0|north|1', 'wall|classroom 0|east|2', 'wall|classroom 0|south|3']⟩
Current Tool to Use: 'get_wall_info'
Arguments: ['wall|classroom 0|west|0', 'wall|classroom 0|north|1', 'wall|classroom 0|east|2', 'wall|classroom 0|south|3']
Example 5.
User Instruction: There is one bedroom in the scene. In the room, there is are two beds. One bed is larger than the other bed.
Previous Constraint Outputs: ["There is one room in the scene, and the Id is 'room 0'.". "The Id of the beds are 'king-size_bed-0 (bedroom 0)', 'bed-0 (bedroom 0)'."]
Current Constraint: In the bedroom, One bed is larger than the other bed.
Tool Sequence: ['get_object_info', 'get_multiview_rendered_object', 'get_multiview_scene_object']
Reasoning behind Tool Sequence: The Id of the beds are 'king-size_bed-0 (bedroom 0)' and 'bed-0 (bedroom 0)', which I can find from the previous constraint outputs. I will use that Id to use the 'get_object_info' tool to get the information of it. I will especially look at the coordinates of the beds. I will also use the object Id as input for the 'get_multivew_rendered_object' and 'get_multiview_scene_object' tools. By using those two tools, I will visually check if the one bed is actually bigger than the other.
Previous Tool Outputs:
Current Tool to Use: 'get_object_info'
Arguments: ['king-size_bed-0 (bedroom 0)', 'bed-0 (bedroom 0)']
User Instruction: {instruction}
Previous Constraint Outputs: {constraint_output}
Current Constraint: {constraint}
Tool Sequence: {tool_sequence}
Reasoning behind Tool Sequence: {reasoning}
Previous Tool Outputs: {tool_output}
Current Tool to Use: {tool_to_use}
Arguments:
