Fsvm2 : ecscCTF 2024
This is the second part of fsvm, if you haven’t checked out the writeup of fsvm yet, then I recommend you to give that a read before reading this.
In this challenge, a distinct set of VM binary and bytecode are provided with few changes in the instruction set of the VM.
Initial Triage
With the exception of a few opcodes being implemented, the decompilation of this binary appears almost identical.
16 registers are used by this virtual machine (reg0 - regf).
I was unable to properly disassemble the emulator after adding the additional opcodes since it appeared to be off by one or two opcodes. It is faster and more efficient to utilize gdb for disassembly since the bytecode is significantly bigger than the previous bytecode.
Let’s first see which opcodes are utilized in this instance.
1 | ['(', ')', '*', '+', '-', '/', '4', '5', '6', '7', '8', ';', '=', '>', '?', '@', 'A', 'B', 'C', 'D', 'F', 'G', 'J', 'K', 'L', 'M', 'N', 'O', 'Q', 'R', 'S', 'T', 'U', 'V', 'W'] |
Opcodes Resolution
Some of the opcodes which are new or implemented for the first time in this bytecode are:
- Opcode “V”: It reads the flag from a file, and the filename is read from
reg0 - Opcode “(“: Increments the value of PC by the value of
rega(Equivalent to Jump instruction in x86) - Opcode “)”: Based on the outcome of the comparison, increases the PC by the value in
regaby comparing the values ofregbandregc. (Equivalent to JZ instruction in x86) - Opcode “+”:
regbandregcundergo a less than operation, and the value of PC is increased in response to the comparison. (Equivalent to JLE instruction in x86)
Building the GDB Script
Following some static and dynamic analysis, I used GDB to write a disassembler. Let’s analyse the disassembly obtained from the first iteration of gdb script .
The beginning of the disassembly reveals that it constructs a filename to read the flag from, and the Opcode “V” reads this filename from reg0. Specifically, the generated filename is flag.txt.
Following this, the code calculates the length of our input flag and saves it in reg1.

This part seems like the length check, because the length of the sample flag that I used in this instance is 48.
So now I know for the fact that the length of the flag should be 37.

Now, let’s re-run the disassembler with the correct flag length.

We can clearly see the difference between the sequence of instructions executed if the flag length is correct
On further analysis, we can observe the following things:
- A number is generated, here
74 - Last character of the input is extracted and stored in
reg1 - Rest of the flag is stored in
reg8

- A loop runs for 74 times, and in each iteration it adds the value of the last character to
reg2 - All that would need to be done is multiply the final character of the input by the number that is produced (74).

Later the entire sum after 74 iterations is stored in reg4
The similar thing happens with all 37 characters of the flag.
At the end, the entire result of reg4 is compared with a value.

Hence, the First loop looks like this:
1 | flag[36]*74 + flag[35]*46 + flag[34]*70 + flag[33]*79 + flag[32]*58 + flag[31]*39 + flag[30]*41 + flag[29]*26 + flag[28]*46 + flag[27]*1 + flag[26]*79 + flag[25]*1 + flag[24]*27 + flag[23]*66 + flag[22]*34 + flag[21]*83 + flag[20]*97 + flag[19]*35 + flag[18]*94 + flag[17]*79 + flag[16]*37 + flag[15]*17 + flag[14]*76 + flag[13]*59 + flag[12]*97 + flag[11]*95 + flag[10]*1 + flag[9]*70 + flag[8]*26 + flag[7]*28 + flag[6]*99 + flag[5]*1 + flag[4]*95 + flag[3]*51 + flag[2]*10 + flag[1]*61 + flag[0]*44 == 172921 |
Since the 2 numbers are not same, the control skips to the end of the program.

Although I was unsure of the number of such comparisons that would occur, I could still bypass that check in order to retrieve all of the accompanying hardcoded integer values and conditions.
The disassembly in that case would look like this .
Additionally, since I am aware of the significance of each instruction, I may print only the cases that include the crucial constant values rather than the complete disassembly.
1 | import gdb |
The final disassembly would look like this .
This shows the final number that each loop checks against, as well as all the constants that are multiplied by each character. There are 37 such checks, which aligns with the length of the flag.
Parsing the constant values
Let’s disassemble that and get all those values.
I eliminated the initial portion of the disassembly that calculates the input flag’s length in order to extract the constants.
Hence, we will perform the extraction on this modified file .
1 | # Extracting the constants |
A constraint solver called z3 would be the best option to solve these contraints and obtain the flag.
So let’s format these values in the z3 script
1 | vals = [74, 46, 70, 79, 58, 39, 41, 26, 46, 1, 79, 1, 27, 66, 34, 83, 97, 35, 94, 79, 37, 17, 76, 59, 97, 95, 1, 70, 26, 28, 99, 1, 95, 51, 10, 61, 44, 48, 74, 4, 92, 1, 26, 42, 94, 78, 44, 80, 62, 61, 27, 80, 48, 38, 46, 32, 92, 98, 66, 4, 7, 6, 61, 96, 71, 3, 32, 70, 94, 72, 60, 45, 9, 45, 64, 75, 63, 9, 40, 87, 26, 72, 30, 23, 49, 78, 66, 92, 23, 34, 42, 22, 7, 42, 12, 79, 5, 73, 27, 83, 88, 35, 1, 37, 28, 65, 30, 65, 67, 42, 1, 75, 85, 45, 73, 29, 53, 51, 90, 76, 37, 44, 7, 14, 47, 85, 57, 84, 5, 63, 96, 34, 44, 32, 47, 36, 25, 55, 28, 92, 47, 92, 48, 19, 7, 29, 2, 29, 52, 43, 57, 4, 11, 70, 95, 97, 49, 84, 80, 70, 58, 26, 7, 8, 67, 8, 87, 25, 62, 6, 90, 20, 92, 11, 99, 39, 61, 47, 55, 31, 67, 37, 46, 17, 91, 40, 70, 49, 17, 31, 29, 44, 32, 19, 6, 16, 7, 12, 58, 18, 1, 55, 22, 59, 12, 86, 74, 76, 15, 40, 65, 40, 36, 70, 23, 87, 32, 70, 32, 92, 79, 63, 99, 38, 38, 84, 80, 48, 100, 85, 71, 46, 73, 78, 96, 11, 49, 6, 32, 93, 74, 35, 6, 37, 83, 42, 86, 12, 10, 3, 26, 93, 57, 27, 55, 55, 76, 43, 52, 15, 41, 21, 83, 40, 1, 24, 69, 73, 55, 88, 94, 83, 15, 48, 22, 41, 58, 25, 35, 63, 63, 45, 39, 84, 38, 83, 84, 30, 80, 1, 94, 62, 56, 91, 9, 96, 63, 3, 3, 37, 31, 5, 90, 79, 10, 35, 35, 77, 13, 58, 8, 37, 29, 23, 61, 83, 9, 12, 95, 95, 57, 98, 11, 48, 87, 87, 75, 43, 51, 34, 44, 51, 6, 80, 75, 10, 63, 100, 44, 24, 79, 69, 98, 57, 48, 81, 53, 29, 43, 25, 14, 75, 11, 1, 26, 7, 24, 77, 52, 37, 64, 43, 45, 85, 3, 73, 73, 15, 55, 95, 76, 96, 42, 24, 91, 89, 59, 21, 65, 74, 79, 10, 77, 74, 58, 98, 84, 39, 36, 45, 86, 44, 39, 90, 81, 88, 33, 37, 52, 70, 42, 19, 39, 77, 53, 70, 8, 38, 69, 58, 34, 10, 47, 100, 13, 90, 21, 69, 36, 80, 15, 31, 36, 6, 95, 46, 76, 35, 30, 45, 38, 25, 9, 13, 52, 94, 99, 100, 54, 84, 18, 36, 23, 55, 34, 22, 37, 64, 50, 57, 76, 63, 99, 99, 33, 92, 35, 57, 83, 30, 86, 76, 79, 48, 54, 20, 28, 8, 36, 70, 83, 89, 21, 7, 63, 46, 5, 65, 42, 58, 17, 99, 97, 22, 45, 43, 90, 55, 5, 95, 32, 12, 82, 15, 2, 86, 77, 31, 79, 61, 1, 1, 32, 25, 37, 4, 17, 47, 81, 69, 94, 31, 59, 91, 60, 68, 30, 15, 100, 64, 62, 78, 89, 34, 51, 8, 72, 90, 35, 62, 17, 86, 2, 51, 38, 61, 85, 95, 55, 87, 21, 18, 19, 11, 61, 72, 38, 30, 33, 83, 9, 40, 60, 40, 14, 100, 76, 54, 19, 6, 91, 43, 13, 7, 38, 62, 76, 29, 64, 18, 51, 73, 14, 7, 97, 70, 24, 56, 68, 62, 57, 19, 46, 48, 18, 9, 22, 67, 12, 97, 20, 51, 71, 83, 51, 42, 21, 91, 15, 81, 26, 87, 25, 61, 48, 1, 50, 42, 46, 26, 88, 61, 18, 52, 79, 50, 86, 97, 4, 76, 27, 40, 63, 32, 79, 45, 83, 2, 7, 47, 32, 58, 57, 92, 47, 32, 7, 68, 78, 50, 52, 73, 98, 57, 29, 15, 22, 69, 27, 71, 57, 59, 77, 13, 95, 54, 97, 52, 82, 42, 86, 42, 91, 87, 84, 41, 57, 43, 30, 98, 46, 74, 22, 31, 44, 28, 16, 68, 70, 34, 88, 92, 36, 100, 98, 46, 38, 49, 70, 26, 1, 21, 20, 35, 60, 6, 71, 82, 58, 80, 25, 86, 74, 70, 98, 12, 33, 97, 36, 28, 56, 11, 58, 47, 38, 30, 52, 70, 93, 26, 83, 52, 5, 36, 44, 48, 11, 58, 91, 57, 62, 49, 47, 76, 14, 71, 78, 97, 14, 80, 43, 98, 5, 13, 40, 27, 92, 82, 10, 6, 29, 42, 7, 10, 42, 42, 40, 92, 53, 3, 55, 27, 62, 94, 39, 1, 21, 88, 39, 28, 7, 30, 98, 96, 16, 91, 99, 66, 46, 63, 16, 5, 68, 85, 62, 51, 25, 69, 12, 48, 30, 86, 43, 14, 99, 88, 94, 51, 24, 37, 21, 5, 83, 5, 15, 51, 11, 57, 57, 83, 69, 90, 78, 41, 72, 80, 37, 7, 99, 7, 7, 94, 12, 7, 14, 81, 6, 35, 85, 100, 36, 47, 53, 84, 63, 87, 6, 72, 85, 88, 33, 79, 2, 63, 12, 31, 18, 31, 33, 60, 50, 47, 45, 48, 28, 79, 9, 42, 68, 35, 14, 12, 42, 15, 67, 66, 47, 13, 25, 63, 64, 45, 28, 10, 98, 32, 14, 3, 55, 42, 19, 31, 68, 54, 45, 62, 40, 34, 88, 60, 33, 16, 37, 79, 14, 68, 28, 2, 35, 36, 10, 71, 45, 18, 2, 57, 72, 31, 73, 63, 43, 64, 30, 92, 35, 59, 54, 1, 53, 38, 67, 17, 55, 59, 98, 41, 71, 96, 30, 60, 4, 99, 7, 98, 50, 94, 24, 66, 40, 41, 89, 26, 65, 23, 36, 26, 31, 37, 96, 63, 89, 100, 51, 60, 3, 24, 69, 65, 98, 10, 13, 15, 7, 96, 54, 25, 97, 11, 69, 52, 86, 6, 69, 13, 17, 73, 33, 72, 2, 87, 55, 54, 69, 70, 8, 100, 7, 77, 48, 18, 15, 68, 96, 46, 38, 100, 89, 13, 21, 48, 41, 7, 4, 2, 55, 83, 16, 62, 96, 28, 30, 97, 18, 39, 8, 23, 81, 61, 60, 77, 23, 81, 45, 95, 11, 45, 96, 77, 21, 73, 39, 64, 66, 57, 40, 23, 62, 46, 74, 61, 77, 73, 82, 30, 53, 14, 66, 11, 79, 99, 75, 31, 58, 56, 39, 34, 75, 25, 6, 72, 71, 95, 45, 35, 79, 75, 59, 21, 55, 47, 58, 55, 78, 28, 45, 2, 5, 49, 37, 67, 13, 44, 29, 64, 97, 6, 34, 49, 63, 10, 32, 59, 12, 68, 29, 65, 52, 86, 40, 32, 57, 17, 90, 98, 96, 41, 92, 80, 3, 12, 41, 56, 88, 22, 54, 28, 26, 68, 47, 91, 48, 28, 3, 69, 49, 78, 42, 67, 81, 34, 49, 11, 71, 25, 19, 2, 23, 61, 29, 39, 77, 38, 99, 91, 16, 19, 3, 77, 58, 65, 44, 7, 33, 79, 45, 51, 28, 39, 17, 9, 24, 78, 10, 47, 73, 3, 70, 33, 70, 25, 93, 66, 24, 83, 29, 7, 4, 18, 28, 41, 45, 21, 56, 66, 14, 51, 4, 7, 26, 23, 48, 2, 65, 8, 33, 70, 47, 98, 7, 75, 85, 8, 24, 68, 25, 20, 76, 98, 27, 25, 32, 98, 94, 42, 95, 80, 77, 89, 17, 87, 89, 66, 3, 31, 95, 6, 99, 99, 98, 85, 70, 48, 40, 65, 37, 79, 44, 32, 55, 6, 93, 2, 39, 43, 60, 85, 1, 19, 50, 100, 61, 36, 80, 26, 44, 29, 9, 82, 72, 66, 36, 80, 36, 44, 24, 10, 17, 19, 69, 51, 86, 88, 95, 73, 96, 13, 85, 15, 87, 19, 100, 51, 96, 41, 60, 50, 42, 34, 32, 92, 29, 96, 77, 37, 49, 48, 5, 72, 84, 11, 33, 87, 22, 93, 100, 59, 26, 21, 29, 97, 10, 86, 10, 9, 3, 49, 59, 47, 25, 57, 34, 25, 95, 29, 6, 5, 80, 8, 98, 76, 90, 33, 34, 18, 26, 84, 76, 99, 1, 86, 87, 69, 21, 42, 65, 57, 71, 59, 15, 70, 15, 61, 73, 88, 76, 2, 61, 89, 47, 28, 47, 52, 33, 39, 13, 15, 5, 33, 28] |
The resultant z3 script:
1 | from z3 import * |
Flag
openECSC{1ts_jus7_4n0ther_vm_582e5a5}
Closing Notes
I enjoyed how this VM was constructed and divided into two parts, each getting harder. This could be a good entry point into the world VM challenges.
Ciao !!
- Title: Fsvm2 : ecscCTF 2024
- Author: Sejal Koshta
- Created at : 2024-04-03 16:30:02
- Updated at : 2024-08-12 22:42:13
- Link: https://k1n0r4.github.io/2024/04/03/Fsvm2-ecscCTF-2024/
- License: This work is licensed under CC BY-NC-SA 4.0.