KolibriOS/kolibrios
13
4
Fork 11
Code Issues 78 Pull Requests 5 Projects Releases Activity

WIP: feat: add NVMe driver #91

Draft
ramenu wants to merge 5 commits from ramenu/kolibrios:main into main
pull from: ramenu/kolibrios:main
merge into: KolibriOS:main
Conversation 4 Commits 5 Files Changed 4 +2329
ramenu commented 2024-08-21 17:26:32 +02:00
Member
Copy Link

This adds the current implementation of the NVMe driver from https://git.kolibrios.org/GSoC/kolibrios-nvme-driver.

Currently, the driver seems to work fine on QEMU and Virtualbox, at least from mine and @Burer's tests. I also believe VMWare works, but shutdown on VMWare doesn't work as expected (see GSoC/kolibrios-nvme-driver#5). Unfortunately, we haven't had any tests on real hardware, which is why I have decided to not include it in the AUTORUN.DAT for now, but if you want to add it anyways then that's fine with me.

Lastly, I want to thank all the mentors, developers, and testers and give credit where credit is deserved. Most importantly, @punk_joker for guiding me on how to implement the driver and answering most of my questions, @Burer for testing the driver, and @Doczom, @dunkaist, and @Sweetbread for helping answering and troubleshooting the miscellaneous problems that I had.

This adds the current implementation of the NVMe driver from https://git.kolibrios.org/GSoC/kolibrios-nvme-driver. Currently, the driver seems to work fine on QEMU and Virtualbox, at least from mine and @Burer's tests. I also believe VMWare works, but shutdown on VMWare doesn't work as expected (see https://git.kolibrios.org/GSoC/kolibrios-nvme-driver/issues/5). Unfortunately, we haven't had any tests on real hardware, which is why I have decided to not include it in the AUTORUN.DAT for now, but if you want to add it anyways then that's fine with me. Lastly, I want to thank all the mentors, developers, and testers and give credit where credit is deserved. Most importantly, @punk_joker for guiding me on how to implement the driver and answering most of my questions, @Burer for testing the driver, and @Doczom, @dunkaist, and @Sweetbread for helping answering and troubleshooting the miscellaneous problems that I had.
👍 2 ❤️ 1
ramenu added 1 commit 2024-08-21 17:26:46 +02:00
dunkaist was assigned by Sweetbread 2024-08-21 17:52:00 +02:00
Sweetbread self-assigned this 2024-08-21 17:52:00 +02:00
Doczom was assigned by Sweetbread 2024-08-21 17:52:00 +02:00
Sweetbread reviewed 2024-08-21 18:06:21 +02:00
Sweetbread left a comment
Owner
Copy Link

You still have TODO comments. Will they be implemented?
Also, I don't think it's worth to add special vim comments

You still have TODO comments. Will they be implemented? Also, I don't think it's worth to add special vim comments
drivers/nvme/command.inc
@ -0,0 +131,4 @@
mov eax, [prp1]
mov dword [esp + SQ_ENTRY.prp1], eax
movzx eax, [fid]
;or eax, 1 shl 31 ; CDW10.SV
Sweetbread commented 2024-08-21 17:59:39 +02:00
Owner
Copy Link

Delete if unnecessary

Delete if unnecessary
drivers/nvme/command.inc
@ -0,0 +203,4 @@
endp
; See page 117-118 of the NVMe 1.4 specification for reference
; INCOMPLETE
Sweetbread commented 2024-08-21 18:01:05 +02:00
Owner
Copy Link

Is the driver still WIP?

Is the driver still WIP?
drivers/nvme/nvme.inc Outdated
@ -0,0 +445,4 @@
sgls dd ?
mnan dd ?
rb 224
subnqn dq ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?
Sweetbread commented 2024-08-21 18:03:30 +02:00
Owner
Copy Link

-> rq 32?

-> `rq 32`?
Doczom reviewed 2024-08-21 18:27:39 +02:00
drivers/nvme/nvme.asm Outdated
@ -0,0 +8,4 @@
;; ;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
format PE DLL native
Doczom commented 2024-08-21 18:27:39 +02:00
Owner
Copy Link

format PE DLL native 0.05

adding this value reduces the number of false positives of antiviruses

format PE DLL native 0.05 adding this value reduces the number of false positives of antiviruses
ramenu commented 2024-08-21 19:09:24 +02:00
Author
Member
Copy Link

You still have TODO comments. Will they be implemented?
Also, I don't think it's worth to add special vim comments

Most of the TODO comments are just future enhancements. The TODO comment you mentioned in particular is an optional command that I was implementing at one point, but stopped when I realized it wasn't important.

Is the driver still WIP?

Currently yes, but all of the features have been implemented. By WIP, I mean it could still be significantly optimized. There are however some parts which I do need to properly check and make sure problems don't occur (see nvme.asm line 267) and the interrupt handler for device conflict. In practice, I don't think the interrupt handler conflict is a problem since the driver checks if any commands have been submitted to the queue anyways, but I'm not certain.

rq 32?

Didn't think about that. Thank you. :)

format PE DLL native 0.05

adding this value reduces the number of false positives of antiviruses

Noted. Thanks.

> You still have TODO comments. Will they be implemented? > Also, I don't think it's worth to add special vim comments Most of the TODO comments are just future enhancements. The TODO comment you mentioned in particular is an optional command that I was implementing at one point, but stopped when I realized it wasn't important. > Is the driver still WIP? Currently yes, but all of the features have been implemented. By WIP, I mean it could still be significantly optimized. There are however some parts which I do need to properly check and make sure problems don't occur (see nvme.asm line 267) and the interrupt handler for device conflict. In practice, I don't think the interrupt handler conflict is a problem since the driver checks if any commands have been submitted to the queue anyways, but I'm not certain. > rq 32? Didn't think about that. Thank you. :) > format PE DLL native 0.05 > > adding this value reduces the number of false positives of antiviruses Noted. Thanks.
ramenu added 1 commit 2024-08-21 19:40:02 +02:00
Sweetbread approved these changes 2024-08-21 19:42:03 +02:00
Doczom approved these changes 2024-08-22 16:50:23 +02:00
Sweetbread requested review from dunkaist 2024-08-22 18:28:43 +02:00
dunkaist was unassigned by Sweetbread 2024-08-22 18:28:49 +02:00
Sweetbread removed their assignment 2024-08-22 18:28:49 +02:00
Doczom was unassigned by Sweetbread 2024-08-22 18:28:49 +02:00
sdongles requested changes 2024-08-22 22:13:38 +02:00
drivers/nvme/macros.inc Outdated
@ -0,0 +26,4 @@
add esp, 12
end if
}
sdongles commented 2024-08-22 21:56:28 +02:00
Member
Copy Link

I think you do not need this file. Use system debug macro like other drivers

I think you do not need this file. Use system debug macro like other drivers
drivers/nvme/nvme.asm Outdated
@ -0,0 +1138,4 @@
mov word [esi + edx], ax ; Write to CQyHDBL
mov word [edi + NVM_QUEUE_ENTRY.head], ax
; Unlock the mutex now that the command is complete
sdongles commented 2024-08-22 22:12:46 +02:00
Member
Copy Link

You decided to leave the spinlock mechanism. It is not a problem. But can you add some to-do comments about replacing it and why we need to do it later? Also, what problem did you face trying to replace it with another mechanism?

It will help to understand a problem to someone who will do some work on the driver.

You decided to leave the spinlock mechanism. It is not a problem. But can you add some to-do comments about replacing it and why we need to do it later? Also, what problem did you face trying to replace it with another mechanism? It will help to understand a problem to someone who will do some work on the driver.
drivers/nvme/nvme.asm Outdated
@ -0,0 +1288,4 @@
inc ecx
cmp ecx, ebx
; TODO: Apply solution given by @punk_joker of checking which device
; generated an interrupt.
sdongles commented 2024-08-22 22:02:09 +02:00
Member
Copy Link

Please add some introduction to the problem it is resolving here.

Please add some introduction to the problem it is resolving here.
ramenu commented 2024-08-23 13:45:48 +02:00
Author
Member
Copy Link

You decided to leave the spinlock mechanism. It is not a problem. But can you add some to-do comments about replacing it and why we need to do it later? Also, what problem did you face trying to replace it with another mechanism?

It will help to understand a problem to someone who will do some work on the driver.

Sure thing. There wasn't really a problem per say, but I was just mostly afraid of making changes which could potentially cause regressions to the already large number of bugs that the driver had. Now that most bugs have been resolved, I'll start working on this once the VMWare issue has been resolved.

As for your other comments, noted. I'll make those changes.

> You decided to leave the spinlock mechanism. It is not a problem. But can you add some to-do comments about replacing it and why we need to do it later? Also, what problem did you face trying to replace it with another mechanism? > > It will help to understand a problem to someone who will do some work on the driver. Sure thing. There wasn't really a problem per say, but I was just mostly afraid of making changes which could potentially cause regressions to the already large number of bugs that the driver had. Now that most bugs have been resolved, I'll start working on this once the VMWare issue has been resolved. As for your other comments, noted. I'll make those changes.
ramenu commented 2024-08-23 14:22:26 +02:00
Author
Member
Copy Link

@sdongles, I found a ugly bug prior to line 1290 which will cause the system to freeze if there's more than one NVMe device registered in the system. I'll add a fix for this later today. Thank you for unintentionally helping me spot that. :)

@sdongles, I found a ugly bug prior to line 1290 which will cause the system to freeze if there's more than one NVMe device registered in the system. I'll add a fix for this later today. Thank you for unintentionally helping me spot that. :)
ramenu changed title from feat: add NVMe driver to WIP: feat: add NVMe driver 2024-08-23 14:22:51 +02:00
ramenu added 3 commits 2024-08-23 18:35:44 +02:00
ramenu changed title from WIP: feat: add NVMe driver to feat: add NVMe driver 2024-08-26 00:49:34 +02:00
ramenu changed title from feat: add NVMe driver to WIP: feat: add NVMe driver 2024-08-26 18:32:43 +02:00
ramenu commented 2024-08-26 18:36:46 +02:00
Author
Member
Copy Link

Nevermind, there's another nasty bug I found which only happens under certain circumstances, see GSoC/kolibrios-nvme-driver#7.

Nevermind, there's another nasty bug I found which only happens under certain circumstances, see https://git.kolibrios.org/GSoC/kolibrios-nvme-driver/issues/7.
This pull request is marked as a work in progress.
View command line instructions.

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u main:ramenu-main
git checkout ramenu-main
Sign in to join this conversation.
Reviewers
No reviewers
Sweetbread
Doczom
dunkaist
sdongles
Labels
Clear labels   
C

   
Category/Applications

   
Category/Drivers

   
Category/General

   
Category/Kernel

   
Category/Libraries

   
Eolite

   
FASM

   
FS

   
HLL

   
Influence/Settings

   
Influence/Text/TYPO

   
IRCC

   
Kernel
  
Kind
Breaking
Breaking change that won't be backward compatible

  
Kind
Bug
Something is not working

  
Kind
Build

  
Kind
Documentation
Documentation changes

  
Kind
Enhancement
Improve existing functionality

  
Kind
Feature
New functionality

  
Kind
Security
This is security issue

  
Kind
Testing
Issue or pull request related to testing

  
Priority
Critical
The priority is critical

  
Priority
High
The priority is high

  
Priority
Low
The priority is low

  
Priority
Medium
The priority is medium

  
Reviewed
Confirmed
Issue has been confirmed

  
Reviewed
Duplicate
This issue or pull request already exists

  
Reviewed
Invalid
Invalid issue

  
Reviewed
Won't Fix
This issue won't be fixed

  
Status
Abandoned
Somebody has started to work on this but abandoned work

  
Status
Blocked
Something is blocking this issue or pull request

  
Status
Need More Info
Feedback is required to reproduce issue or to continue work

No Label
C
Category/Applications
Category/Drivers
Category/General
Category/Kernel
Category/Libraries
Eolite
FASM
FS
HLL
Influence/Settings
Influence/Text/TYPO
IRCC
Kernel
Kind
Breaking
Kind
Bug
Kind
Build
Kind
Documentation
Kind
Enhancement
Kind
Feature
Kind
Security
Kind
Testing
Priority
Critical
Priority
High
Priority
Low
Priority
Medium
Reviewed
Confirmed
Reviewed
Duplicate
Reviewed
Invalid
Reviewed
Won't Fix
Status
Abandoned
Status
Blocked
Status
Need More Info
Milestone
Clear milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
No Assignees
4 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: KolibriOS/kolibrios#91
No description provided.
Delete Branch "ramenu/kolibrios:main"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?